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More SQL Server 2008 T-SQL 
Improvements 
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the performance benefits now—with this walk-through 
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Find SQL Server 


Cost Savings 


| these trying economic times many SQL 
n Server shops are looking for ways to save 
money. Most organizations want to press forward 
with their existing workloads, but they want to do 
it cheaper. So what are some of the best ways to 
cuts costs for a SQL Server installation? 


Lower Development and 
Departmental System Costs 

Try running your development systems on SQL 
Server Developer Edition. At $50 per processor 
and no CAL costs the Developer Edition is sig- 
nificantly cheaper than full-featured SQL Server 
editions. Even though you can’t use SQL Server 
Developer Edition to run your production work- 
loads it makes good economic sense for your 
development servers to run it. 

Next, look for savings in your departmental 
systems. Lightly loaded systems or servers with 
limited users are great candidates to convert to the 
free SQL Server Express. Often overlooked, SQL 
Server Express is a very capable relational data- 
base server. It’s limited to 1 CPU, 1GB of RAM, 
and a maximum of 4GB of storage per user da- 
tabase, but that configuration can support a good 
number of concurrent users for lightly taxed de- 
partmental systems. Remember that SQL Server 
Express can act only as a subscriber and not as a 
publisher, so you can’t use replication publication 
on these systems. SQL Server 2008 Express sup- 
ports the new synchronization services, but older 
SQL Server implementations typically won’t be 
using these services. 

Many organizations buy the Enterprise edi- 
tion because they don’t want to spend time mak- 
ing decisions about features or worrying about 
the capability to apply cool new SQL Server tech- 
nologies to their projects. However, if you're not 
using the features you’re paying for, you're leaving 
money on the table. 

Although SQL Server 2008 Enterprise Edition 
is the most expensive edition of SQL Server, it pro- 
vides some money-saving features—for example, 
database compression and the Resource Gover- 
nor. Database compression can reduce the storage 
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required by your server and your backup media, 
as well as reduce the time to perform backups. Re- 
source Governor can stave off the need to upgrade 
servers by limiting the CPU usage by queries that 
are low priority or poorly designed. This lets criti- 
cal queries run without interference. 


Try Consolidation and 

Virtualization 

Server consolidation enables you to process the 
same workload on fewer systems. You can con- 
solidate servers in a number of different ways— 
by combining workloads, by combining multiple 
SQL Server instances, or through virtualization. 
Combining workloads and/or instances can re- 
duce the number of servers required as well as the 
number of SQL Server licenses you need. How- 
ever, server consolidation requires you to perform 
due diligence to ensure that different applications 
don’t adversely affect one another. For more in- 


formation, vist www.windowsitpro.com/ebooks/ 


index.cfm?fuseaction=ebook&ebid=bacle8b0- 
640a-460f-8313-lab25a86a8cc. 


Virtualization can also help you reduce the 
number of servers you need, but SQL Server li- 
censes are still required unless you’re using SQL 
Server Enterprise Edition. SQL Server Enter- 
prise Edition allows an unlimited number of 
virtual SQL Server instances if you’ve licensed 
all the processors in the host. For more informa- 
tion on licensing, check out www.microsoft.com/ 
sqlserver/2005/en/us/Special-Considerations 


.aspx. 


Hey! You’re Already Saving Money 
Remember, of all of the enterprise database 
offerings, SQL Server is the only one that in- 
cludes business intelligence (BI) functional- 
ity and user friendly features such as SQL 
Server Reporting Services right out of the box 
with no additional costs. Do you have any 
SQL Server cost saving tips that you'd like to 
share? If so, drop me a line at motey@sqlmag 
.com or letters@sqlmag.com. SQL 
InstantDoc ID 101526_ 


Michael Otey 


(motey @ sqlmag.com) is technical director 
for Windows IT Pro and SQL Server Maga- 
zine and author of Microsoft SQL Server 

2008 New Features (Osborne/McGraw-Hill). 
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SOLAGNE 
e’re l0! 


We're Going Strong! 
And We Love SQL Server Pros! 


Join us in celebrating 
milestones along the way! 


It’s been 10 YEAR I: 
E Microsoft releases version 7.0 of 
SQL Server 2000 (1998) 


amazing years ‘A 
| E Microsoft releases SQL Server 2000 (September 1998) 


since the launch of 


SQL Server Magazine 


Tal February 1998, . [I Microsoft releases SOLXML 2.0 (2002) 
and thanks to 
you - our loyal 


E Microsoft releases SQL Server 2005 


and passionate (November (2005) 


community of 
YEAR 10: 

- W Microsoft launches SQL Server 2008 
(February 2008) 

: W Microsoft release SQL Server 2008 
RTM code (August 2008) 


SQL Server pros — 
we're stronger 


than ever! 


Thanks to all SQL Server Magazine 
community members! 


Past articles and more on our Web site at sqlmag.com 


Riding Out the IT Storm 


EY morning, the headlines get worse. Now 
spanning far beyond its origin in the mort- 
gage industry, the recession has turned its cor- 
rosive spotlight on every other sector you could 
name. But two of those damaged sectors cut close 
to home for the staff and contributors of SOL 
Server Magazine: The publishing industry and the 
IT industry. 

The publishing industry’s problems are not new 
to this recession. But the recession will hasten the 
overhaul of this industry that has been coming for 
years. Particularly in the business-to-business pub- 
lishing arena, the cost of paper, printing, and post- 
age for producing the magazine you hold in your 
hands has risen astronomically. At the same time, 
advertisers want results-oriented marketing. We 
can demonstrate results when a reader clicks on an 
online advertisement. But we can’t prove that you 
took action because of an ad in the magazine. So 
in some months we’ll have fewer ads and inevitably 
the magazine sometimes will be thinner until media 
planners discover anew the branding and awareness 
value of print magazines. 

In the meantime, we’re looking at the powerful 
ways we can bring database professionals techni- 
cal content and industry analysis in multiple for- 
mats—including print magazines, Web seminars, 
in-person events, conferences, virtual events, vid- 
eos, blogs, and forums. We are reinventing the way 
we deliver content to you so that the format better 
matches the information. Like Microsoft, we can 
give you “on premises” products—the print maga- 
zine, books and DVD resources, and in-person 
events. But we also deliver services “in the cloud” 
with online training and technical information 
resources. 

The first step toward this new information de- 
livery is the relaunch of our website, which should 
be a done deal this spring. The new platform will 
allow us more flexibility in delivering content and 
will feature more screen area devoted to editorial 
content and related visuals. You'll also notice new 
tools for helping you connect with the database 
community, including revamped blogs, forums, 
and social networking features. 
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Our new web strategy will allow our editors to 
produce and acquire high-quality content, plus 
editors will point you to the best technical ar- 
ticles, training events, and other resources on the 
web. We’ll also bring you more content “from the 
trenches” as we engage with user groups across the 
world. 

We're redoubling our efforts to bring you prac- 
tical information so you can do your current job 
better or acquire new skills. These resources in- 
clude a new online store for technical resources, 


www.left-brain.com, a series of web-based learn- 


ing seminars (www.sqlmag.com/events), and SQL 


Server-focused how-to screencasts at our video 
site, www.ittv.net. In addition to helping you boost 
your technical skills, we’re planning activities to 
help you find your next job—watch for news on 
our site about a web-based job fair coming this 
summer. 

Our focus on skill-boosting and career develop- 
ment reflects the sober state of this industry: Con- 
sulting organization Challenger, Gray & Christ- 
mas calculated that the computer industry ranked 
third among industries suffering job losses, with a 
whopping 22,330 layoffs in January alone. In defi- 
ance of the bleak statistics, chip manufacturer Intel 
bravely forged ahead with innovation in February 
by announcing a $7 billion investment in manufac- 
turing facilities for its 32-nanometer technology. In 
a National Public Radio interview, Intel CEO Paul 
Otellini said that the reason for Intel’s boldness 
was simple: The company is betting that technol- 
ogy will lead us out of this recession. 

“We believe that people will continue to want to 
buy computers,” Otellini said. “If your computer 
broke tonight when you went home, you wouldn’t 
wait for the recession to end to buy a new com- 
puter. It’s an indispensible part of your life.” 

Otellini’s conviction that technology is what will 
pull us out of the recession is inspiring—and, we 
hope, accurate. In the meantime, we’ll all do well 
to learn new skills so we'll be ready to ride the wave 
back up—and celebrate SOL Server Magazine’s 15" 
birthday! SQL 
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(michele.crockett@ penton.com) helped 


launch SQL Server Magazine in 1999, has 
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is currently editorial and custom strategy 
director of Windows IT Pro, SQL Server 
Magazine, and System iNEWS. 
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Partition Overview For Database partitiontest 


[dbo].[order_history] (PK_order_history, CLUSTERED) 


Partition Scheme: partscheme2(OrderData) 
Partition Function: partfunc2(datetime) - RANGE RIGHT 


Find Out What’s Going On 


Inside Your Partitions 


ne area in which SQL Server Management 
Studio (SSMS) is sorely lacking is partitioned 
tables. There are no built-in GUI tools to create parti- 
tion functions or partition schemes. And after you’ve 
hammered out your T-SQL queries to set them up, 
well, that’s pretty much it. Although you can use the 
$PARTITION statement to get a rough idea of what’s 
in each partition, this statement doesn’t provide infor- 
mation about file-group usage and other 
important details. 
Although I don’t have a silver bullet for 
creating and maintaining partitions with a few 


the code at 


B | C | D 
index_type column 
CLUSTERED OrderData 
CLUSTERED OrderData 
CLUSTERED OrderData 


PK_order_history 
PK_order_history 
PK_order_history 


clicks, I do have a script—PartitionOverview.sql—that 
will make it a lot easier to get an idea of what’s going on 
inside your partitions. You can run PartitionOverview 
.sql in any database to get detailed information about 
your partitioned 
tables and in- 
dexed views. Par- 
titionOverview 
„sql isn’t a stored 


Partition Included Boundary Filegroup Rows % of Rows 

} FGt o 000% | procedure, but 

2 7/1/2008 12:00:00 AM FG3 0 0,00 % ` 

3 8/1/2008 12:00:00 AM FG2 o 090% | you could easily 
Figure 2 turn it into one. 


Excerpt of sample 
results from Partition- 
Overview.rdl 
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Share your SQL Server 
code, comments, discov- 
eries, and solutions to 
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number. We edit submis- 
sions for style, grammar, 
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—kKaren Bemowski, 
senior editor 
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To use PartitionOverview.sql, all you need to do 
is run it. No tweaking is necessary. So what will you 
see when you run it? Figure 1 shows an excerpt from 
a sample results grid created with PartitionOverview 
.sql. Although Figure 1 shows only several columns 
and rows, you'll get a result set with 13 columns and 
one row for each partition. The 13 columns are as 
follows: 

e object—Identifies the table or view that has been 
partitioned. 

e index—Specifies the index that has been 
partitioned. 

e index_type— Notes whether the index is clustered 
or nonclustered. 

e column—lIdentifies the column the index is parti- 
tioned on. 

e partition_scheme—Notes the partition scheme used 
to partition the index. 

e partition_function—Specifies the underlying parti- 
tion function that defines the partition scheme. 

e parameter_type—Identifies the type of parameter 
required by the partition function. 


partition_scheme 
partscheme2 
partscheme2 
partscheme2 


e partition_number—Provides the partition number 
in the index. 

e boundary_type—Specifies whether the partition 
function uses RANGE RIGHT or RANGE LEFT. 

e included_boundary_value—Notes the partition 
function’s boundary value within the partition. 
(Note that you'll always have one NULL for each 
index. So, for example, a partition function with 
three boundaries will create a partition scheme with 
four partitions.) 

e filegroup—tdentifies the file group that the partition 
is stored on. 

* rows—Specifies the number of rows in 
the partition. 

* percent_of_rows—Shows the percentage 
of rows in the index. 


PartitionOverview.sql contains a few other 
columns that are commented out. Most of them 
are for internal SQL Server IDs (e.g., data space ID, 
parameter ID) that aren’t directly related to partitions 
but might be of interest to those DBAs who want to 
further develop the tool. 

While writing this script, I think I discovered why 
I never found similar tools in the past. Collecting 
all the information requires 13 JOIN operations of 
various views. So, the code doubles as a learning tool 
for those DBAs curious about how views like sys 
.destination_data_spaces and sys.index_columns relate 
to partitioning. 

But wait, there’s more! As an excuse to better 
familiarize myself with SSMS’s Custom Reports fea- 
ture, I used PartitionOverview.sql as the foundation 
for an easy-to-read report named PartitionOverview 
dl. PartitionOverview.rdl provides the same data as 
PartitionOverview.sql, but the data is formatted so that 
the information is easier to read, especially if you have 
a lot of tables and indexes. Figure 2 shows an excerpt 
from a sample report. 

To use PartitionOverview.rdl, you need to open 
SSMS, right-click your database, drill down to Custom 
Reports, and choose the PartitionOverview.rdl file. You 
don’t need to tweak PartitionOverview.rdl. 

You can download PartitionOverview.rdl, 
PartitionOverview.sql, a sample PartitionOverview.rdl 
report, and a sample PartitionOverview.sq] results grid 
by going to www.sqlmag.com, entering 101051 in the 
InstantDoc ID text box, and clicking the 101051.zip 
hotlink. PartitionOverview.sql and PartitionOverview 
.rdl work on SQL Server 2005 and later. SOL} 
—Dave Britten, DBA, Great Lakes Computer Source 
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rior to SQL Server 2008, the database engine 

tried to provide balanced performance to all 

concurrent users, so you had little control 
over the priority assigned to users or workloads. In 
addition, many organizations have hundreds (or even 
thousands) of applications, and DBAs have little 
knowledge of which applications are being used to 
access their databases. The problem is made worse by 
the use of fixed SQL Server logins for a large number 
of users of applications. 

SQL Server 2008 introduces the Resource Gover- 
nor, which provides you with a degree of control over 
your resources. Resource Governor can help prevent 
performance problems and somewhat identify the 
applications being used. Although Resource Gover- 
nor currently has several limitations, it’s sufficient for 
dealing with many common resource problems. In this 
article, Pll show you how to manage your CPU and 
memory resources by creating resource pools, assign- 
ing applications to resource pools, and using classifier 
functions to assign sessions to workload groups. 


Sample CREATE 
RESOURCE POOL Commands 


CREATE RESOURCE POOL LowPriorityAppsPool 
WITH (MAX_CPU_PERCENT = 29); 


CREATE RESOURCE POOL MediumPriorityAppsPool 
WITH (MAX_CPU_PERCENT = 69); 


CREATE RESOURCE POOL HighPriorityAppsPool 
WITH (MAX_CPU_PERCENT = 199); 


EOS SF 


Resource Governor works by controlling the CPU 
and memory allocated to a session. You can specify 
the maximum and minimum percentages of CPU 
and memory that an application can use, as well as 
the maximum degree of parallelism permitted. To do 
so, create broad pools of resources to allocate sessions 
into and set the minimum and maximum percentages 
on them using the CREATE RESOURCE POOL 
command. For example, the commands in Listing 1 
create three resource pools that are limited based on 
CPU percentage. The CPU percentages used in these 
commands aren’t hard limits. For instance, if a single 
low priority application is the only application run- 
ning on the system, it will be allocated total use of 
the CPU. The governing of resources based on the 
CPU percentage occurs only in times of overlapping 
concurrent demands. 

In Resource Governor there’s a hard-coded maxi- 
mum limit of 20 pools, 18 of which are available for 
configuration. Of the remaining two pools, one is allo- 
cated to internal processes, such as cleanup operations, 
and the other is the default pool used for sessions not 
assigned to other pools. Therefore, you shouldn't cre- 
ate too many resource pools too early and should care- 
fully consider which pools will be needed. 


Once you’ve grouped the available resources, you can 
group the applications that will be assigned to these 
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REGAIN CONTROL of your SQL Server 
systems by assigning resources to workloads 


resource pools by creating workload groups using the 
CREATE WORKLOAD GROUP command. List- 
ing 2 shows some sample CREATE WORKLOAD 
GROUP commands. 

In Listing 2, five workload groups were created 
and assigned to three resource pools. The Access and 
Excel applications are kept in separate pools, in case 
you want to change the resources allocated to one 
versus the other in the future. Poorly behaving Access 
applications and applications that we can’t identify 
will be given few resources. Applications that we have 
identified as critical to our business will be given the 
highest priority on resources. 


Viewing Your Resource 
Governor Configuration 

Once you've created the workload groups and as- 
signed them to resource pools, they don’t start work- 
ing immediately. To determine what else needs to be 
configured to get Resource Governor up and running 
you can view the current Resource Governor configu- 
ration, which is provided in a new dynamic manage- 
ment view. Executing the command 


SELECT * FROM sys.dm_resource_governor_ 
configuration; 


produces the output shown in Figure 1. (Note that 
this output is shown in the Results tab in SQL Server 
Management Studio—SSMS.) The value of 1 in the 
left column is the output row number. 

This output tells you two things. First, the 
is_reconfiguration_pending flag means that you 
need to execute the command 


ALTER RESOURCE GOVERNOR RECONFIGURE; 


before your changes will take effect. After you 
run this command, the is_reconfiguration_pend- 
ing value is zero. Second, the value of zero for 
the classifier_function_id means no classifier 
function has been assigned. (As a side note, 
I feel this column should return NULL, 
not zero, for this value.) A classifier func- 
tion is used to determine which workload 
group a session should be placed in. PI 
show you how to use classifier functions in 
a moment. 

Next, you can see the resource pool 


Figure 3 
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configuration by executing the command 


SELECT * FROM sys.dm_resource_governor_ 


resou rce_pool S; 


which returns the output shown in 
Figure 2. Until you execute the recon- 
figuration command, only the internal 
and default pools will appear in this 
output. You can see the workload pool 
configuration by using the command 


SELECT * FROM sys.dm_resource_ 
governor_workload_groups; 


which returns the output shown in 
Figure 3. 


Classifying 

Applications 
The final step to configuring Resource 
Governor is identifying the sessions 
that you want to control. You can do 
so by using a classifier function, which 
assigns a session to a spe- 


LISTING 2: Sample 
CREATE WORKLOAD 


GROUP Commands 


CREATE WORKLOAD GROUP 
UnidentifiedApplications 
USING LowPriorityAppsPool; 


CREATE WORKLOAD GROUP 
Wel1BehavedAccessApplications 
USING MediumPriorityAppsPool; 


CREATE WORKLOAD GROUP 
PoorlyBehavedAccessApplications 
USING LowPriorityAppsPool; 


CREATE WORKLOAD GROUP 
Wel1BehavedExcelApplications 
USING MediumPriorityAppsPool; 


CREATE WORKLOAD GROUP 
CriticalCorporateApplications 
USING HighPriorityAppsPoo1; 


cific workload group, as 
Listing 3, page 14, shows. |4 o 

Classifier functions 
must return a value of 
type SYSNAME, which 


Figure | 


Satisics_san_time 
2009-02-03 07:11:58.863 
2008-02-03 07:11:58.863 


T Fente | Messages 
classifier_function_id 


is_reconfiguration_pending 
1 


Output showing the current Resource Governor 
configuration 


total_cpu_usage_ms cache_memory_ 
21370 6576 
45375 26776 


2009-02-03 13:12:07.370 
2009-02-03 13:12:07.370 
2009-02-03 13:12:07.370 


Figure 2 


WellBehavedExcelApplications 
Critical CorporateApplicatons 


pool 
1 

2 

256 
257 
256 
257 
258 


Output showing the workload group configuration 


| id statisics_start_ime 
2009-02-03 07:11:56.853 0 
2009-02-03 07:11:58.863 
2009-02-03 13:12:07.370 
2009-02-03 13:12:07,370 
2008-02-03 13:12:07.370 
2009-02-03 13:12:07.370 
2009-02-03 13:12:07.370 


Output showing the resource pool configuration 


Sotal_request_count 


10875 
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LISTING 3: Code to Assign 
Sessions to Workload Groups 


CREATE FUNCTION dbo.UserClassifier O 
RETURNS SYSNAME WITH SCHEMABINDING 


AS 
BEGIN 


RESOURCE GOVERNOR 


is currently an alias for NVAR- 
CHAR(128). For the purpose 
of this article, I have based the 
decision about which workload 
group a session should go into 


DECLARE @WorkloadGroup SYSNAME 


on the application name func- 


The LOGINPROPERTY() function now includes 
two properties (DefaultDatabase and DefaultLan- 
guage) that can be used in classification functions. In 
addition, the CONNECTIONPROPERTY() func- 
tion provides access to the network transport and 
protocol being used for the connection, as well as 


= N'UnidentifiedApplications'; 


SET @WorkloadGroup = CASE APP_NAME() 


tion (APP_NAME()). This func- 


WHEN N'Excel_Link_BR3' THEN 


N'Wel1BehavedExcelApplications' 


WHEN N'Excel_Link_GR8' THEN 


N'Wel1BehavedExcelApplications' 


WHEN N'HR_Access_G13"' THEN 


tion returns the value that was 
specified in the connection string 
from the application. I have a 


N'Wel1BehavedAccessApplications' 
WHEN N'HR_Access_G19' THEN 

N'PoorlyBehavedAccessApplications' 
WHEN N'Corp_HR_ Maintenance’ THEN 

N'CriticalCorporateApplications' 
ELSE N'UnidentifiedApplications' 


END; 
RETURN @WorkloadGroup; 
END; 


Resource 
Governor 
works really 
well for 
long-running 
queries that 
eat up 
resources 
over a long 
period of 
time. 


T Resets 9) Messages 


(No column name) 


particular liking for this option 
as it lets you make classification 
decisions based on which appli- 
cation is running. There are sev- 
eral other options you can use 
when classifying sessions, which 
I'll describe later. 

Applications are often built 
without the application name being specified in the 
connection string, but it’s important to add the ap- 
plication name to the connection string or to specify 
the value in the application configuration files. This 
practice is also useful outside of Resource Governor 
because the application name appears in traces cre- 
ated by SQL Server Profiler and can be used to filter 
events within the traces. Having access to application 
names via the connection strings also assists with 
tracing access to SQL Server when trying to isolate 
an application-related problem. However, some less 
well designed applications have hard-coded connec- 
tion strings that don’t lend themselves to this ap- 
proach. Hard-coding connection strings is a practice 
that should be avoided. 

Also, I have found classifying applications by 
name to be a good starting point for identifying un- 
known applications that are connected to a server. 
The beauty of Resource Governor in this situation is 
that the unknown applications would be placed in a 
low priority resource pool, and it’s likely that the us- 
ers involved would then quickly identify the applica- 
tions they’re using. This capability lets you determine 
who is actually using your systems. 

Although the APP_NAME() function was used in 
Listing 3, many other functions can be used to clas- 
sify sessions, such as HOST_NAME(), SUSER_ 
NAMEQ(, SUSER_SNAMEQ), IS_SRVROLEMEM- 
BERQ), and IS-MEMBER(). If you decide to use 
either HOST_.NAME() or APP_NAME() as the 
classifier function, be aware that it’s possible for these 
functions to be 
altered by users. 


| Microsoft SQL Server Management Studio - Query ] 


In practice, how- 


Figure 4 


Output from the APP_ 
NAME() 
classifier function 


14 April 2009 


ever, I have found 

APP_NAMEZ( to work very well. 
In addition to these functions, you can use sev- 
eral properties when making classification decisions. 


details of the authentication scheme, the local IP ad- 
dress, the TCP port, and the client’s IP address. The 
ORIGINAL_DB_NAMEQ() function is also useful 
because it returns the name of the database that was 
provided when the session first connected, as opposed 
to the default database. These properties let you make 
classification decisions based on any of these val- 
ues. For example, you could assign a connection to 
a workload group based on which database the user 
typically uses. 

It’s important to test the classifier function before 
putting it to use; otherwise your system can become 
unresponsive. For example, you can test the APP_ 
NAME() classifier function in SSMS by executing 
the command 


SELECT APP_NAME() ; 


which returns the output shown in Figure 4. Because 
SSMS wasn’t on the list of applications that the clas- 
sifier function knew about, you shouldn't be surprised 
that executing the command 


SELECT dbo.UserClassifierQ; 


returns the output shown in Figure 5. If youenabled the 
classifier function without further modification, SSMS 
wouldrun in the low priority application resource pool. 

You can also use a classifier function to look up 
the workload group name in a database table, instead 
of hard-coding all application names into the func- 
tion. Typically, performance isn’t greatly affected 
when you access a table to look up the workload 
group, because the table isn’t large and is quickly 
cached, and the classification decisions that the func- 
tion is used to make are taken only when a connection 
is first established. 

You can now implement the classifier function us- 
ing the following command: 


ALTER RESOURCE GOVERNOR 
WITH (CLASSIFIER_FUNCTION = dbo. 
UserClassifier) ; 


Then you must reconfigure Resource Governor to put 
the pending configuration into operation. You can do 
so using the command 


ALTER RESOURCE GOVERNOR RECONFIGURE; 


At this point, your Resource Governor configu- 
ration is complete. You then need to make sure it’s 
working as it should. If you did a poor job of test- 
ing your classifier function, the system could become 
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unresponsive to new logins and you might need to 
use the dedicated administrator connection (DAC) 
to gain access to SQL Server. If you haven’t config- 
ured the DAC, you can restart the server in single- 
user mode. When running in single-user mode, your 
connection isn’t subject to Resource Governor, so you 
can disable Resource Governor until the situation is 
corrected. 

Should you need to change the classifier function, 
you'll need to first disable Resource Governor be- 
cause you can’t change it while Resource Governor is 
in use. You can disable the current classifier function 
by executing the following command: 


ALTER RESOURCE GOVERNOR 
WITH C(CLASSIFIER_FUNCTION = NULL); 


As a general rule, Resource Governor changes 
should be implemented by the most senior DBA 
within your organization, given the problems that can 
arise if Resource Governor is misconfigured. That’s 
also the reason I’ve used T-SQL scripts for implemen- 
tation in this article, even though you can perform 
these operations directly in SSMS. The CONTROL 
SERVER permission is required to change the con- 
figurations, and the VIEW SERVER STATE permis- 
sion is required to view the configuration. Classifier 
function failures can be detected via events in the ap- 
plication log on the server. 


Other Techniques for 

Monitoring and Configuring 
Resources 

You can also monitor the CPU resources being al- 
located by SQL Server via the System Monitor 
(perfmon.exe) tool. Note that when you add counters 
that you want to monitor, the SQLServer:Resource 
Pool Stats object shows instances of each of the con- 
figured resource pools, as shown in Figure 6. Similar 
instance-related selections have been added to the 
SQLServer: Workload Group Stats counter. Relevant 
values can also be obtained by querying the sys.dm_ 
os_performance_counters view. 

To enable more detailed monitoring, additional 
events have been added to SQL Server, including the 
CPU Threshold Exceeded, PreConnect:Starting, and 
PreConnect:Completed events. If you want to control 
Resource Governor programmatically using .NET 
code, a ResourceGovernor class has been added to 
SQL Server Management Objects (SMO). 

For the purposes of this article, I focused on 
maximum CPU percentages when configuring re- 
source pools because you must be very careful about 
how you configure minimum percentages. It’s easy to 
greatly limit the resources on a system by configuring 
the minimum values too high, particularly minimum 
percentages for memory. 
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RESOURCE GOVERNOR 


Caveats 

You might be wondering about the limi- 
tations I mentioned earlier. I can hear 
people saying “OK Greg, this feature 
looks great. What’s the catch?” First, re- 
source pools and workload groups can’t be dropped 


TI Renis 1) Messages 
(No column name) 


Output from 


while they’re in use. No big surprise there, except they  4b0-UserClassifier() 
could be in use for quite a while on busy systems. Also, 
@ Reliability and Performance Monitor ~ — =e 4 


@ Add Counters 


Available counters 
Select counters from computer: 
<Local computer > 


SQLServer: SQL Errors 
SQLServer:SQL Statistics 
SQLServer:SSIS Pipeline 10.0 
SQLServer:SSIS Service 10.0 
SQLServer:Transactions 
SQLServer:User Settable 


Instances of selected object: 


| default 
|HighPriorityAppsPool 
internal 

| LowPriorityAppsPool 
MediumPriorityAppsPoo! 


[<alinstances> 


| UnidentifiedApplications | 
Figure 5 


Counter 


you might think that you can use Resource Governor 
to drop the priority of that pesky query that you’ve 
just noticed is currently killing system performance. 
However, you can’t use Resource Governor to do 
that because the workload groups and resource pools 
were allocated when the session connected. In addi- 
tion, you can’t use Resource Governor to allocate 
priorities amongst your OLTP applications. Short 
OLTP queries aren’t good candidates for being gov- 
erned this way. Resource Governor works really well 
for long-running queries that eat up resources over a 
long period of time. 

Finally, at nearly every site that I work on, SQL 
Server is I/O bound, not CPU or memory bound. 
SQL Server systems that are CPU bound tend to have 
a limited set of problems, such as plan cache pollu- 
tion problems that are causing constant recompila- 
tion. Although limiting memory can help starve out 
smaller queries in some data warehousing scenarios, 
what I would like to see is the ability to govern file I/O 
on a file-by-file basis. Despite these shortcomings, I 
think you'll find that Resource Governor is a great 
addition to your toolkit and an excellent option to 
help you regain control of your systems. SQL 
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Composable DML 
SQL Server 2005 introduced support for the OUTPUT 
clause, allowing modification statements such as 


ast month I covered a few small T-SQL 
enhancements in SQL Server 2008, including 
the enhanced VALUES clause, support for ISO 


week as a date part, and enhanced binary-character 
conversions. This month I'll cover four additional 
enhancements: composable DML, the ability to 
declare and initialize variables in the same statement, 


INSERT, UPDATE, and DELETE to return output 
with information from modified rows. The initial 
implementation of the OUTPUT clause doesn’t let 
you filter output rows; namely, if the modification 


compound assignment operators, and more reliable 
object dependency information. 


MORE on the WEB 


Download the listings at 
InstantDoc ID 101345. 


will return 50 rows. The only flexibility is that 

you can select which attributes or expressions, 

based on existing attributes, you want to return 
from the modified rows. In addition, you 
can control the target of the OUTPUT 
clause (caller, table, or both). Because 
you can’t filter output rows, if you want 

IF OBJECT_ID('dbo.Orders', 'U') IS NOT NULL DROP TABLE dbo.Orders; to store a subset of the output rows in a 

CREATE TABLE dbo.Orders 

( target table, you must apply a two-step 


statement affects 50 rows, an OUTPUT clause 


LISTING |: Code to Create Orders 
and OrdersAudit Tables 


SET NOCOUNT ON; 
USE tempdb; 


orderid INT NOT NULL, i ae j 

e Or M process: You have to output the TOWS 
empid INT NOT NULL, into a staging table, then insert into the 
custid  VARCHAR(5) NOT NULL, 

a INT e NOT NULL, final target table the filtered rows that you 
CONSTRAINT PK_Orders PRIMARY KEY(orderid) 


ie need from the staging table. 
co SQL Server 2008 enhances support 


a a 26), 
R bys 
6", 22), 


INSERT INTO dbo.Orders r 
Grid, orderdate, empid, custid, aty) for the OUTPUT clause with a new 
VALUES atur ; i 
TE, ARAR, J A D: feature called composable DML. This 
(19801, '20061224', 1, 'A', 12), feature lets you insert into a target table a 
(19005, '20061224', 1, 'B', 20), b fr dfi dificé 
(40901, "20070109" 4. ‘A’. 4g). subset of rows returned from a modifica- 
(19006, ° 20970118", 1, 'C', 14), tion statement. The composable DML 
(20001, '20079212', 2, 'B', 12), rae Á 
(40005, '20080212', 4, 'A', 10), feature is implemented in the form of an 
(20982, '20080216', 2 i 
(30003, '20980418'| 3 INSERT SELECT statement, possibly 
(39004, ' 20960418", 3 with a WHERE clause. The FROM 
(30987, '20069907', 3, 'D', 36); 
clause of the SELECT statement defines 
IF OBJECT_ID('dbo.OrdersAudit', 'U') IS NOT NULL à derived table that is based on anodic 
fication statement with an OUTPUT 


DROP TABLE dbo.OrdersAudit; 
CREATE TABLE dbo.OrdersAudit 


(( A 
auditlsn INT NOT NULL IDENTITY, clause. To demonstrate this feature, create 
loginname SYSNAME NOT NULL DEFAULT (SUSER_SNAME()), : i a : A A 
posttime DATETIME NOT NULL DEFAULT (CURRENT_TIMESTAMP) , the Orders and OrdersAudit tables by This Required Reading 
orderid INT NOT NULL, running the code in Listing 1. i 
orderdate DATE NOT NULL, 8 8 Is sponsored by 
empid INT NOT NULL, Suppose that you need to purge QUEST 
custid VARCHAR(5) NOT NULL, o nl: < 
aay ‘ivi NOT NULL. orders placed prior to 2007, and you SOFTWARE’ 
CONSTRAINT PK_OrdersAudit PRIMARY KEY(audit1sn) need to audit deleted rows only if they 

J; 


represent orders placed by customers A 
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T-SQL IMPROVEMENTS 


TABLE |: Content of OrdersAudit Table 


auditlsn loginname posttime 


íl DOJO\Gandalf 2009-02-12 19:55:04.017 
2 DOJO\Gandalf 2009-02-12 19:55:04.017 
3 DOJO\Gandalf 2009-02-12 19:55:04.017 


LISTING 2: Code to Illustrate 
Composable DML with MERGE 


INSERT INTO dbo.CustomersAudit(custid, companyname, phone, address) 


SELECT custid, companyname, phone, address 
FROM (MERGE INTO dbo.Customers AS TGT 
USING dbo.CustomersStage AS SRC 
ON TGT.custid = SRC.custid 
WHEN MATCHED THEN 
UPDATE SET 
TGT.companyname = SRC.companyname, 
TGT.phone = SRC.phone, 
TGT.address = SRC.address 
WHEN NOT MATCHED THEN 
INSERT (custid, companyname, phone, address) 


VALUES (SRC.custid, SRC.companyname, SRC.phone, SRC.address) 


OUTPUT $action AS action, 
inserted.custid, 
inserted. companyname, 
inserted.phone, 
inserted.address) AS D 

WHERE action = 'INSERT'; 


LISTING 3: Code to Declare 
and Initialize Variables 


DECLARE 
@v1 AS INT = 19, 
@v2 AS DATE = SYSDATETIME(), 


@v3 AS NVARCHAR(2Q@8) = (STUFF( 


(SELECT N',' + QUOTENAME (COLUMN_NAME) 
FROM INFORMATION_SCHEMA.COLUMNS 
WHERE TABLE_SCHEMA = N'dbo' 

AND TABLE_NAME = N'Orders' 


ORDER BY ORDINAL_POSITION 


orderid orderdate empid custid qty 


10001 2006-12-24 1 A 12 
30001 2006-08-02 3 A 10 
30007 2006-09-07 3 D 30 


No other querying elements are 
allowed in the outer statement 
(e.g., joins, grouping). Also note 
that similar restrictions that apply 
to the OUTPUT clause also apply 
to composable DML. That is, the 
target table can be a permanent 
table, temporary table, or table 
variable. But the target can’t be a 
table expression (e.g., a view), have 
triggers, participate in a primary 
key-foreign key relationship, or 
participate in merge replication 
or updatable subscriptions for 
transactional replication. 
SQL Server 2008 supports the 
OUTPUT clause and compos- 
able DML with the new MERGE 
statement. For an example of using 
composable DML with the MERGE 
statement, run the code in Listing 2. 
This example demonstrates how you 
can audit output rows from a MERGE 
statement—but only those rows that 


FOR XML PATH('')), 1, 1, N'')); 


SELECT @v1 AS v1, @v2 AS v2, @v3 AS v3; 


were affected by an INSERT action; 
output rows that were affected by 
other actions won't be audited. 


TABLE 2: Values of Variables after Initialization 


vl v2 v3 
10 2009-02-12 


or D. Instead of implementing a two-step process, you 
can implement the task in one step using composable 
DML like so: 


INSERT INTO dbo.OrdersAudit(orderid, orderdate, 
empid, custid, qty) 
SELECT orderid, orderdate, empid, custid, qty 
FROM (DELETE FROM dbo.Orders 
OUTPUT deleted. * 
WHERE orderdate < ‘20070101’) AS D 
WHERE custid IN (‘A’, ‘D’); 


Table 1 shows the contents of the OrdersAudit table 
after running this code. 

Note that the current implementation of compos- 
able DML allows only an INSERT SELECT state- 
ment against the derived table with a WHERE clause. 


[orderid],[orderdate],[empid],[custid], [qty] 


Declare and Initialize 
Variables 

SQL Server 2008 introduces the ability 
to declare and initialize variables in the 
same statement. The initialization value can be any 
self-contained expression that returns a scalar value. 
For example, Listing 3 demonstrates declaring and 
initializing variables with a constant, a function, and 
a scalar subquery. Table 2 shows the content of the 
variables after the initialization. 


Compound Assignment 
Operators 

Like the previous enhancement I described, com- 
pound assignment operators are another small syn- 
tactical improvement. Prior to SQL Server 2008, if 
you wanted to assign a value to a column, variable, or 
parameter, and that value was the result of applying 
an operator (e.g., addition or subtraction) to that 
column and another operand, you had to refer to that 
column again in the expression. For example, if you 
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TABLE 3: Values of Variables 


after As 
ER 


w0 |i) 2 8 | dabi 


wanted to add 10 to a column called coll in an 
UPDATE statement, you had to specify the fol- 
lowing assignment in the statement’s SET clause: 
coll = coll + 10. 

With compound assignment operators you 
don’t have to repeat the target name; rather, 
you specify: coll += 10. SQL Server 2008 sup- 
ports compound assignment operators wherever 
assignment is supported, such as in an UPDATE 
statement, MERGE statement, SET statement, 
and assignment SELECT statement. Compound 
assignment operators include arithmetic operations 
(+=, -=, *=, /=, %=), bitwise (|=, &=, ^=), and even 
string concatenation (+=). The following code dem- 
onstrates using the various compound assignment 
operators in the context of an assignment SELECT 
statement: 


DECLARE 
@a AS INT = 10, @b AS INT = 10, @c AS INT = 
16, @d AS INT = 10, @e AS INT = 10, 


LISTING 4: Code to Create Objects for 


SQL Dependencies Example 


USE tempdb; 


IF OBJECT_ID('dbo.Proc1', 'P') IS NOT NULL DROP PROC dbo.Procl; 
IF OBJECT_ID('dbo.Proc2', 'P') IS NOT NULL DROP PROC dbo.Proci; 


T-SQL IMPROVEMENTS 


Qf AS INT = 10, @g AS INT = 10, Gh AS INT 
= 10, 
@i AS VARCHAR(10) = ‘abc’; 


SELECT 
@a += 2, @b -= 2, @c /= 2, Gd *= 2, Ge %= 
2, -- arithmetic 
Gf |= 2, @g & 2, Gh A= 2, -- bitwise 
@i += ‘def’; -- 
concatenate 


SELECT @a AS a, @b AS b, @c AS c, @d AS d, @e 
AS e, Qf AS f, @g AS g, Gh AS h, @i AS i; 


Table 3 shows the values of the variables after the 
assignment. 


SQL Dependencies 

Prior to SQL Server 2008, object dependency 
information that SQL Server kept track of was 
unreliable. The main reason for this unreliability 
was the combination of support for deferred 
name resolution and the fact that dependency 
information was tracked based on object IDs. 
Deferred name resolution means that you can 
create objects that depend on other objects even 
if the other objects don’t exist. For example, you 
can create a stored procedure that refers to a 
table that doesn’t exist. Since a 
nonexistent object doesn’t have 
an object ID yet, when you 
create an object that refers to a 
nonexistent object, SQL Server 
has nothing to record as depen- 


IF OBJECT_ID('dbo.V1', ‘V’) IS NOT NULL DROP VIEW dbo.V1; 


IF OBJECT_ID('dbo.T1', ‘U’) IS NOT NULL DROP TABLE dbo.T1; 


GO 


CREATE PROC dbo.Procl 
AS 

EXECC EXEC dbo.Proc2'); 
EXEC dbo.Proc3; 


GO 

CREATE PROC dbo.Proc2 
AS 

SELECT * FROM dbo.T1; 

GO 

CREATE TABLE dbo.T1(col1 INT); 
GO 

CREATE VIEW dbo.V1 

AS 

SELECT coll FROM dbo.T1; 
GO 


dency info. Even if you later 
create the referenced object, it’s 
too late, because SQL Server 
records dependency informa- 
tion only when creating refer- 
encing objects if the referenced 
objects exist. 

SQL Server 2008 records 
dependency information based 
on name references. So even if 
you create an object that refers 
to another object that doesn’t 
yet exist, SQL Server can parse 


TABLE 4: Output of Query Against 
sys.sql_expression_dependencies 
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TABLE 5: Output of Query Against 
sys.dm_sql_referenced_entities 


objschema objname minorname class 
dbo Tl NULL OBJECT_OR_COLUMN 
dbo Tl coll OBJECT_OR_COLUMN 


TABLE 6: Output of Query Against 
sys.dm_sql_referencing_entities 


objschema  objname Class 
dbo Proc2 OBJECT_OR_COLUMN 
dbo V1 OBJECT_OR_COLUMN 


and record the name references. If an object doesn’t 
exist when you query the dependency information, SQL 
Server can show you the dependencies by name, without 
the ID. If the object is created later, SQL Server can then 
return the ID as well. 

Note that SQL Server 2008 doesn’t provide depen- 
dency information for dynamic code or Common Lan- 
guage Runtime (CLR) code. It provides information for 
dependencies that appear in static code. You get infor- 
mation about schema-bound and non-schema-bound 
objects, cross-database, and even 
cross-server dependencies. 

To demonstrate querying the 
new object dependency infor- 
mation, run the code in Listing 
4, page 19, to create several 
objects in the tempdb database 
for test purposes. SQL Server 
2008 introduces three objects 
that return dependency infor- 
mation: the sys.sql_expression_dependencies catalog 
view and the sys.dm_sql_referenced_entities and sys 
.dm_sql_referencing entities dynamic management 
functions (DMFs). 

The sys.sql_expression_dependencies view pro- 
vides object dependencies by name. It replaces the 
older sys.sql_dependencies view. For example, the 
following query shows all dependencies in the current 
database: 


SELECT 

OBJECT_SCHEMA_NAME(referencing_id) AS 
srcschema, 

OBJECT_NAME(referencing_id) AS srcname, 
referencing_minor_id AS srcminorid, 
referenced_id, 
referenced_schema_name AS tgtschema, 
referenced_entity_name AS tgtname, 
referenced_minor_id AS tgtminorid 

FROM sys.sql_expression_dependencies; 


This query produces the output in Table 4, page 19. 


SQL Server 
2008 includes 
numerous small 
enhancements. 


Notice that the dependencies in the dynamic 
SQL code weren't identified, whereas all 
dependencies that appear in the static code 
were identified. Also notice that the depen- 
dency of Proc! on the nonexistent Proc3 was 
identified by name, but since Proc3 doesn’t 
exist, the referenced_id attribute for Proc3 
is NULL. 
The sys.dm_sql_referenced_entities DMF provides 
all entities that the input entity references. Those 
are the entities that the input entity depends on. For 
example, the following code returns all entities that 
dbo.V1 depends on: 


SELECT 
referenced_schema_name AS objschema, 
referenced_entity_name AS objname, 
referenced_minor_name AS minorname, 
referenced_class_desc AS class 
FROM sys.dm_sql_referenced_entities(‘dbo.V1’, 
‘OBJECT’); 


This code returns the output in Table 5. The output 
tells you that dbo.V1 depends on the table dbo 
.T1 and the column dbo.T1.coll. 

The sys.dm_sql_referencing_ 
entities DMF provides all enti- 
ties that reference the input 
entity. Those are the entities 
that depend on the input entity. 
For example, the following code 
returns all entities that depend 
on dbo.T1: 


SELECT 
referencing_schema_name AS objschema, 
referencing_entity_name AS objname, 
referencing_class_desc AS class 
FROM sys.dm_sql_referencing_entities(‘dbo.T1’, 
‘OBJECT’); 


This code returns the output in Table 6. The output 
shows you that dbo.Proc2 and dbo.V1 depend on 
dbo.T1. 


Little Changes Can Bring Big 
Improvements 
SQL Server 2008 includes numerous small enhance- 
ments. Some changes, such as compound assignment 
operators and the ability to declare and initialize 
variables in the same statement, are just syntactical 
improvements that let you shorten your code a bit. 
Other changes, such as composable DML and the new 
object dependency information, are more significant, 
providing improved performance and reliability. EAH 
InstantDoc ID 101345 
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Using 


SQL Server 2008 


FILESTREAM Storage 


New feature lets you store 
BLOB data as full-blown files 


Server 2008’s FILESTREAM 
storage offers exciting new 
performance improvements 
for storing BLOB data as full-blown files managed by 
the OS. But as great as the FILESTREAM storage fea- 
ture is, it’s not exactly trivial to use and requires some 
legwork to set up correctly. Let’s look at what you need 


to do to set up, configure, and code to take advantage 
of FILESTREAM storage. 


Why Use FILESTREAM Storage? 
Conventional wisdom has long maintained that storing 
and retrieving files and images within a relational data- 
base incurs significant performance penalties. That logic 
seems sensible, but I had always wondered whether it 
was based on empirical evidence or mere superstition. 
In 2006, Microsoft Research published an excellent 
white paper that tackled this subject in detail. Its authors 
conclude that databases could learn a thing or two from 
file systems, which are much better at managing frag- 
mentation than are relational database engines. Inter- 
estingly enough, they also determined that file systems 
could learn from database engines by virtue of how 
they handle smaller bursts of data. (See “To BLOB or 
not to BLOB: Large Object Storage in a Database or 


File System” at research.microsoft.com/research/pubs/ 


view.aspx?msr_tr_id=MSR-TR-2006-45.) 

The rough translation: File systems generally work 
best with blocks of data greater than 1MB in size, while 
databases work best with blocks of data less than 256K 
in size. To address this situation, SQL Server 2008 offers 
highly performant storage and retrieval of smaller bursts 
of data and adds FILESTREAM storage, letting SQL 
Server work closely with the file system to achieve the 
best performance from both worlds. 


FILESTREAM Storage Hurdles 

Setting up FILESTREAM storage correctly, however, 
can cause a few headaches. For example, security 
considerations in distributed environments require a 
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bit more planning to make sure that client applica- 
tions are able to take advantage of streamed files that 
are proxied through SQL Server. Similarly, because 
FILESTREAM storage represents a joint interac- 
tion between SQL Server and the host OS, setting 
up FILESTREAM storage on your servers requires 
admin permissions at the OS level. Furthermore, if you 
want to take advantage of Win32 streaming capabili- 
ties, you need to use Windows integrated security (as 
SQL Server logins obviously interact with the under- 
lying file system). These minor limitations, however, 
help to safeguard your infrastructure and data. 

A bigger hurdle is that neither SQL Server Man- 
agement Studio (SSMS) nor SQL Server itself really 
has any UI, or native way, to let you stream 
the contents of a file into a table that’s been 
marked with the FILESTREAM attribute 
on one of your varbinary(max) columns. 
In other words, if you’ve got a .jpg file that 
you want to store within SQL Server, there’s no native 
functionality to convert that image’s byte stream into 
something that you could put, for example, into an 
INSERT statement. 

Accordingly, you have to use middle-tier and client 
applications to leverage this new storage functionality. 
The problem with accessing BLOB data from within 
client and middle-tier applications is that the transac- 
tional choreography and overhead needed to safely and 
securely stream file contents in and out of the native 
file system via SQL Server adds a bit more complexity 
than you'd normally see when working with queries 
that involve simple data readers. 


FILESTREAM Storage Benefits 

Those hiccups aside, FILESTREAM storage is worth 
using in many scenarios. Besides its performance benefits, 
it also lets you exceed the 2GB limit previously associated 
with storing BLOBs, so you can store BLOBs as large as 
the OS, or file system, permits. FILESTREAM storage 
also provides transactional safety when storing files 
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Figure | 


Configuring 
FILESTREAM storage at 
the OS level 


and offers increased 
file manageability— 
including the 
obvious benefit of 
file backup and 
restore functionality. 
Other great benefits 
include the ability 
to leverage replica- 
tion, log shipping, 
and the power of 
full-text indexing 
directly against 
files stored and 
managed by SQL 


Server. (However, operations against FILESTREAM 
file groups aren’t supported by database snap- 
shots or database mirroring; see SQL Server 2008 


Books Online “Using FILESTREAM with other 


SQL Server Features” 


technet.microsoft 


.com/en-us/library/bb895334.aspx?ppud=4. ) 


Configuring Your Server for 

FILESTREAM Storage 

Given that the file system directly manages 
FILESTREAM data, with SQL 


LISTING l: Code to Configure 
SQL Server for FILESTREAM 
Access 

EXEC sp_configure 'filestream access level', 2 


GO 
RECONFIGURE 


LISTING 2: Code to Create a 


FILESTREAM-Enabled Database and Table 


CREATE DATABASE FSTest 


PRIMARY 
( NAME = Main, FILENAME = 'D:\SQLData\FSTest_main.mdf'), 
FILEGROUP FileStreamGroup1 
CONTAINS FILESTREAM 
( NAME = FSFiles, FILENAME = 'D:\SQLData\FSTest_files') 
LOG ON 
( NAME = Archlogl, FILENAME = 'D:\SQLData\FSTest_log. 1df') 
GO 


USE FSTest 
GO 


CREATE TABLE FileSamples ( 
[FileId] UNIQUEIDENTIFIER ROWGUIDCOL NOT NULL UNIQUE, 
[FileName] VARCHAR(2@) NOT NULL, 
[FileData] VARBINARY (max) FILESTREAM DEFAULT (Øx) 


LISTING 3: Code to Enable TSQL 
Support for FILESTREAM Storage 


INSERT INTO FileSamples 
SELECT NEWID(), 
"fakefile.txt', 
CAST(N'Some bogus "file" data' AS VARBINARY (MAX) ) 


BEGIN TRANSACTION 

SELECT [FileData] .PathName() , 
GET_FILESTREAM_TRANSACTION_CONTEXTQ)AS [Context] 

FROM FileSamples 

WHERE [FileName] = 'fakefile.txt' 

ROLLBACK 


Server handling file names, paths, 
and even security (by proxy), the 
first thing you need to do when 
using FILESTREAM storage 
is grant SQL Server permission 


to work directly 
with the host OS’s 
file system. Because 
this is an OS-level 


change, enabling 
FILESTREAM 
storage requires 


administrative rights 
on the host server. 
As you can see in 
Figure 1, to enable 
FILESTREAM 
functionality you 
open SQL Server 
Configuration 
Manager, then 


access the Properties page 
for the individual SQL 
Server instance that you 
wish to enable. As part of 
this configuration process, 
you specify what kinds 
of access and streaming 
rights are available to SQL 


Server and remote clients. 
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When it comes to enabling FILESTREAM func- 
tionality, you have three choices: 0, which disables 
FILESTREAM support for this instance; 1, which 
enables FILESTREAM functionality for T-SQL 
access; and 2, which enables FILESTREAM support 
for T-SQL and Win32 access. 

After you make the change from within SQL Server 
Configuration Manager, a new share is created on the 
host system with the name specified. This isn’t a typical 
share, though, and is intended only to allow very low- 
level streaming interaction between SQL Server and 
authorized clients. As such, Microsoft recommends that 
only the service account used by the SQL Server instance 
have access to this share. Moreover, because this change 
takes place at the OS (or service) level, you can’t enable 
it from within SQL Server, and you need to restart your 
SQL Server instance for the change to take effect. 

After you restart, you also need to enable SQL 
Server to utilize this storage option. Run sp_configure 
with the appropriate filestream access level (0, 1, or 2) 
to meet your needs. For my sample application I set the 
value to the most permissive option, 2, which you can 
see in Listing 1. 


Configuring Databases for 
FILESTREAM Storage 

After configuring FILESTREAM storage at the OS 
and SQL Server levels, you can start creating tables to 
house your documents. The key to taking advantage 
of FILESTREAM storage is to decorate a chosen 
varbinary(max) field with the FILESTREAM attri- 
bute as I’ve done in Listing 2. As long as you have cre- 
ated a specialized FILESTREAM File Group within 
your database, the data stored in this column won't 
persist to pages or extents managed by SQL Server. 
Instead it will stream to the file system where you can 
take advantage of the increased performance when it 
comes to larger file sizes. 

Note in Listing 2, however, that the FILEGROUP 
that I’ve specified for use with FILESTREAM storage 
(with the CONTAINS FILESTREAM attribute) 
doesn’t actually point to a file such as files.ndf. Instead, 
it points to a folder, which is where files and file data 
will actually end up being stored. 

FILESTREAM-enabled tables also require the 
existence of a UNIQUE ROWGUIDCOL to support 
storage interaction, which I’ve handled by using the 
FileId column in my sample table. Likewise, in the code 
displayed in Listing 2 I’m also specifying a default of (0x) 
on my FileData column. This default causes SQL Server 
to create a new, blank file within the FILESTREAM 
storage folder for Win32 streaming clients to access and 
use as a path when attempting to upload file contents. 
Without this default creation of a “blank” file, new 
files would have to be created by streaming serialized 
contents directly through T-SQL before they could be 
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SQL SERVER 2008 FILESTREAM STORAGE a i 


accessed via Win32 client [EF FLESTREAM Browser -E 
applications. a 

Listing 3 shows inkata tt 
typical FILESTREAM | Sr 
use where only T-SQL is 
involved. The INSERT 
statement creates a Epot Selected Fle Ae 
new row in the table porcine 
(where the bogus file tbid Rie 
contents are actually Be Source Path = 
persisted to disk as a eon 
file). Then, the SELECT 
statement uses the new Figure 2 


GET_FILESTREAM_ 
TRANSACTION_ 
CONTEXTO) built-in function and the new .Path- 
Name() function to get a transaction context, or ID. 
This transaction context is then handed off to a Win32 
client application to access the contents of the file in 
question at the FileData.PathName() specified. 


Client Applications 

and FILESTREAM Storage 

For a real-world example of using FILESTREAM data, 
Ive created a very simple app called FILESTREAM 
Browser WinForms, which Figure 2 shows, to 


A FILESTREAM-enabled client application 


display the contents of the 
FileSamples table created 
earlier. This simple applica- 
tion also lets users upload 
files from their desktop (or 
elsewhere) to SQL Server 


LISTING 4: Code to Acquire a 
Transaction Context 


Guid fileId = Guid.NewGuid() ; 
SqlConnection conn = new SqlConnection(connString) ; 


conn.Open() ; 
SqiCommand insert = new SqlCommand( 
"INSERT INTO FileSamples ([FileId], [FileName]) " + 
"VALUES (@FileId, @FileName)", conn); 
insert. Parameters.Add("@FileId", 
Sq1DbType.UniqueIdentifier).Value = fileld; 
insert.Parameters.Add("@FileName", 
Sq1DbType.VarChar, 20).Value = fileName; 
insert. ExecuteNonQuery() ; 


SqlTransaction fsTx = conn.BeginTransaction(); 


Sq]Command getTransaction = new Sq]Command( 
"SELECT [FileData].PathName(), " + 
"GET_FILESTREAM_TRANSACTION_CONTEXT() " + 
"FROM FileSamples " + 
"WHERE FileId = @FileID", conn); 

getTransaction.Transaction = fsTx; 

getTransaction.Parameters.Add("@FileId", 
Sq1DbType.UniqueIdentifier).Value = fileld; 


SqlDataReader contextReader = 
getTransaction.ExecuteReader (CommandBehavior 
-SingleRow) ; 

contextReader.Read() ; 

string filePath = contextReader.GetString(@) ; 

byte[] transactionId = (byte[])contextReader[1]; 

contextReader.Close(); 


FILESTREAM storage or download a file placed in 
FILESTREAM storage to whatever path they specify. 

In the code accompanying this article, you'll 
see that UI logic has been relegated to Forml.cs 
while FILESTREAM interactions have all been 
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LISTING 5: Code to Stream File Contents with a SqIFileStream Object 


using (FileStream fs = File.OpenRead(path)) 


using (SqlFileStream sqIFS = new SqlFileStream(filePath, transactionId, FileAccess.Write)) 
i 


byte[] buffer = new byte[512 * 1924]; 
int location = fs.Read(buffer, Ø, buffer.Length); 


while (location > 9) 


sqlFS.Write(buffer, Ø, location); 


location = fs.Read(buffer, Ø, buffer.Length) ; 


i 
i 
3 


fsTx.Commit(); 
conn.Close() ; 
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placed within a helper class found in the 
FileStreamSample.cs file. This class, in 
turn, contains three methods: One method 
pulls back a list of the current files in the 
FileSamples table, and two methods allow 
downloads and uploads as needed. Fur- 
thermore, since files are identified by the 
UNIQUEIDENTIFIER FileId column, 
logic in all three methods is closely centered 
on these GUIDs as handles to help direct 
operations. 

In Listing 4, page 23, code from my 
upload method creates a new file by firing 
off an INSERT statement that populates meta- 
data for the file that will be uploaded. The code 
then creates a new transaction, enlists a command 
object within the transaction (to gain a transaction 
context), and executes a SqlDataReader connection 
to pull back the full path (in SQL Server) to the 
file that will be uploaded, along with a transaction 
context ID that can be used to initiate the Win32 
file-streaming process. By using a context ID and 
the actual FILESTREAM storage path for the new 
file, the code can then initiate a straight file-streaming 
operation by using the SystemData.SqlTypes.Sql- 
FileStream class, which debuted in .NET 3.5 SP1. 

Then, as you can see in Listing 5, I simply create 
a new System.IO.FileStream object to read my file 
locally and buffer bytes along in 512K chunks (an 
arbitrary selection) to my SqlFileStream object until 
there are no more bytes to transfer. At this point, I 
can close the transaction and my file has been suc- 
cessfully uploaded to SQL Server as file data instead 
of normal varbinary(max) data. (Note, however, 
that since we're actively streaming file contents to a 
Win32 process, we have to use integrated security, 
because SQL logins can’t generate the needed tokens 
to access the underlying file system.) 

Download operations follow the same para- 
digm of creating a new transaction, then grabbing 
a transaction context ID and the path for the file 
to stream to your client. However, when it comes 
to the streaming operation with downloads, you 
do everything in reverse—pulling data from a Sql- 
FileStream object into a buffer and pushing it into 
a local FileStream object until there are no more 
bytes left to transfer. 


Increase Performance 
Using the approach I outline above, you can get better 
performance for operations involving large files and 
still take advantage of SQL Server's ability to provide 
high performance processing of small files. In this 
way, you get the best of both worlds: high perfor- 
mance and relational database integrity. SQL 
InstantDoc ID 101388 
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Removing 
Forwarded Records 
from Heap Tables 


Uncover a hidden SQL Server performance killer 


you've checked all the likely suspects, 
KN Q but your SQL Server is still performing 

poorly. Before you put that capital 
request in for more horsepower, think about a hidden 
performance killer you might have missed—forwarded 
records. Forwarded records are a type of fragmenta- 
tion that’s hard to detect and harder to resolve. By 
automating the discovery and resolution of forwarded 
records, you can eliminate this performance killer from 
your databases. This article walks you through how to 
use a stored procedure that’s designed to intelligently 
detect and eliminate forwarded records, as well as log 
the effect of their removal. 


How Forwarded Records Are 
Created 

In essence, forwarded records are “fragmentation” 
within a permanent or temporary heap table. (For- 
warded records are found only in heap tables.) Heap 
tables, the most simple storage arrangement, consist 
of unorganized data pages. Rows are inserted into a 
heap wherever there happens to be space. Because a 
heap table is unordered, the data pages aren’t linked 
together in any way. When a heap table is accessed for 
information and no clustered index is available to sup- 
port requests, the table might be scanned inefficiently, 
especially if forwarded records exist. 

A forwarded record occurs when a heap table (either 
permanent or temporary) is created that includes vari- 
able length fields. When a batch is called to load data 
into the table, some of the variable length fields might 
be populated with a short value (e.g., the letters A or 
N). A second batch might be called to transform the 
data in this field into something more user-friendly 
(e.g., A = Acceptable, N = Not Acceptable) but that 
is a larger value than was initially used. This is exactly 
what creates the forwarded records. 

When a larger value is updated in a variable length 
field in a record, SQL Server will first try to expand 
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the row of the existing page on the chance that there’s 
enough room to continue to use the original page. If 
that fails and SQL Server can’t find an existing page 
with enough room for the larger value, a new page 
has to be created for the record. The data is moved to 
the new page and is assigned a new Relative Identi- 
fier (RID). The old page now contains a forwarding 
pointer (i.e., forwarded record) that tells SQL Server 
to jump to the new RID. The new record location 
also has a back pointer to the old record. Forwarded 
records prevent non-clustered indexes on heap tables 
from being updated with the RID of the new row. The 
overhead of updating indexes is worse than replacing 
the old records with a pointer. But the drawback to 
forwarded records is that their presence could result 
in a large amount of I/O. For each forwarded record, 
SQL Server has to jump to the target page, 
then back to the original one. This is an inef- 
ficient way to read a row and wastes space. 
Although forwarded records can occur 
on all heap tables, they don’t always, and 
you can take steps to alleviate them. For example, you 
could initially populate tables with dynamic columns 
with long values. This approach would initially set 
the record to a large enough size to accommodate 
most updates and create minimal forwarded records. 
Updating a dynamic column from a long value to 
a shorter value won't create any forwarded records. 
Developers should be very aware of how temporary 
tables are created and populated. Often, no one is aware 
of the existence of forwarded records in temporary tables 
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LISTING |: Finding Forwarded Records 


SELECT DB_NAME(database_id) as DatabaseName, 
object_id, 
OBJECT_NAME(OBJECT_ID,database_id) as TableName, 
forwarded_record_count, 
index_type_desc 
FROM sys.dm_db_index_physical_stats (DB_ID(N'FRTest'), 
OBJECT_ID(N'dbo.FRTable'), NULL , NULL, 'DETAILED') 
GO 
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REMOVING FORWARDED RECORDS 


TABLE |: Forwarded Records in FRTable Before Running 


FixForwardedRecs 
DatabaseName  object_id § TableName 
FRTest 21575115 FRTable 


because the objects are destroyed when batches are com- 
pleted. Web Listing 1 (www.sqlmag.com, InstantDoc ID 
101572) shows how forwarded records are created. 


Finding Forwarded Records 
Now that you know how forwarded records are cre- 
ated, let’s look at how you find forwarded records and 
determine if they’re causing poor performance. Use 
the dynamic management view (DMV) sys.dm_db_ 
index_physical_stats, as shown in Listing 1, page 25, 
to find forwarded records. The results from this DMV 
(displayed in Table 1) show that there are now 24,142 
forwarded records in FRTable. If FRTable is called in 
a query from a user’s web application, that user would 
start to experience slow response because SQL Server 
had to do many more reads to satisfy the request. 
After running DBCC DROPCLEANBUFFERS 
and DBCC FREEPROCCACHE, you can check STA- 
TISTICS IO for the query SELECT * FROM FRTable 
before the forwarded records are created to show that 
it performed 75 logical reads, 0 physical reads, and 
21 read-ahead reads. After the forwarded records are 
introduced, the same query performs 24,432 logical 
reads, 0 physical reads, and 46 read-ahead reads. This 
is a 32,576 percent increase in logical reads and a 119 
percent increase in read-ahead reads, which translates 
to much more work for SQL Server. 


forwarded_record_count index_type_desc 
24142 HEAP 


Just for effect, let’s update our variable length field 
(i.e., nvarchar(50) column) with a larger value by using 
the following command: 


UPDATE FRTable SET Expandable = 
REPLICATE(N’x’ , 48) 
co 


If you check the sys.dm_db_index_physical_stats 
DMV again, you can see that the number of forwarded 
records has increased to 26,885. There are now more 
rows than before that have forwarded records. If you 
check STATISTICS IO for the query SELECT * 
FROM FRTable again you'll find that SQL Server 
performed 27,332 logical reads, 0 physical reads, and 
262 read-ahead reads. (Don’t forget to run DBCC 
DROPCLEANBUFFERS and DBCC FREEPROIC- 
CACHE before each attempt.) 


Removing Forwarded Records 
Defragging or rebuilding indexes doesn’t resolve for- 
warded record fragmentation. The following are the 
only three ways to remove forwarded record fragmen- 
tation from a heap table: 

e Add a permanent clustered index. 

¢ Temporarily add a clustered index and then remove it. 
e Move all of the data into another table (e.g., tempo- 


LISTING 2: Creating the FR_LOG Table 


USE ADMINDB 


eee en One ere 
/* Script: FFR_DDL.sql */ 
/* Purpose/Description: Creates log table used in */ 
$ FixForwardedRecs stored procedure to 
/*Notes: FR_LOG table is used to log forwarded record removal cy 
/* information y 
/* CHANGE HISTORY: */ 
/* Date Modified Created By Description of Modification */ 
/* e a +43) a a SS Se et a ee eS e Se */ 


JEEE E II I I e de e IIIT II I e e e III I I I I I IAI III I e e e III III I I II IAAI III I I III e e e AI III AAAI III I IIIA] 


IF EXISTS (SELECT * FROM sys.objects WHERE OBJECT_ID = OBJECT_ID(N'FR_LOG') AND type = (N'U')) 
DROP TABLE FR_LOG 
le) 
CREATE TABLE dbo.FR_LOG ( 
[rowid] [INT] IDENTITY (1, 1) NOT NULL , 
[dbname] [varchar] (128) NULL, 
[objectid] [INT] NULL , 
[objectname] [VARCHAR] (255) NULL 
[schemaname] [VARCHAR] (128) NULL, 
[before_frcount] [INT] NULL , 
[after_frcount] [INT] NULL , 
[seconds_elapsed] [INT] NULL , 
[datedefrag] [SMALLDATETIME] NULL 
) ON [PRIMARY] 
le) 
CREATE CLUSTERED INDEX [CIX_FR_LOG] ON [dbo]. [FR_LOG]([rowid]) ON [PRIMARY] 
le) 
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TABLE 2: Forwarded Records in FRTable After Running FixForwardedRecs 


rowid dbname schemaname  objectid 
1 [FRTest] dbo 21575115 


rary table, table variable, permanent table), truncate 


the original table, and reload the data. 


In my experience, resolving forwarded records 


using one of the three known methods previ- 

ously listed is a slow, manual process. Because 

Tm not a fan of manual processes, I created a 

stored procedure designed to do it all for you. 

The procedure, which uses the first two methods 

I noted, corrects forwarded records without 

permanently changing the table structure and 

without unloading and reloading data in a table. 

The stored procedure does the following: 

1. Provides a dummy clustered index name 
and dummy auto incrementing ID column 
name 

2. Finds tables with forwarded records 

3. Checks for the presence of an auto 
incrementing ID column 

4. Creates a clustered index 
a. If there’s an existing auto incrementing ID 

column, create a dummy clustered index 
on the existing auto incrementing ID 
column. Then drop the dummy clustered 
index when you're done. 

b. If there’s no existing auto incrementing 
ID column, create one using the dummy 
auto incrementing ID column name and 
create the dummy clustered index on 
the new column. First drop the dummy 
clustered index and then drop the dummy 
auto incrementing ID column when you're 
done. 

5. Logs the tables with forwarded records and 
the quantity found before and after the clus- 
tered index is applied 


The objects created by the stored procedure 
are designed to be placed in an administra- 
tive database, so create a database named 
ADMINDB. Then create the FR_LOG table 
using Listing 2. 

Now create the stored procedure using the 
code in Web Listing 2. When you execute the 
stored procedure, you have to supply the data- 
base name that’s going to be checked for for- 
warded records. Because the stored procedure 
is located in an administrative database and not 
installed into each user database, you must use 
three-part naming to access the correct data. 

To call the stored procedure, use 
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objectname before frcount after_frcount seconds elapsed  datedefrag 
[FRTable] 26885 0 0 


2008-04-15 11:33:00 


EXEC ADMINDB.dbo.FixForwardedRecs @dbname = 
‘FRTest’ 


j 

A 
Somé€times thg 
S@t Serv rÆ 


Usually, it’s harder to pinpoint. 
Amazing what you can accomplish once 


A smarter solution makes everyone 
look brilliant. 


SOFTWARE 
Download your FREE trial of Confio Ignite” at www.confio.com/sqimag 
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LISTING 3: Capturing the STATISTICS IO Before Removing Forwarded Records 


set statistics io on 

go 

select AddressLinel, AddressLine2, City, PostalCode 
from dbo.AddressHeap 

where City = 'Bothell' 

go 


-- Stats before removing the forwarded records. 
Scan count 1, logical reads 9877, physical reads Ø, read-ahead reads 9, lob logical reads Ø, lob physical reads Ø, lob read-ahead reads ø 


LISTING 4: Capturing the STATISTICS IO After Removing Forwarded Records 


set statistics io on 


go 
select AddressLinel, AddressLine2, City, PostalCode 
from dbo.AddressHeap 

where City = 'Bothell' 

go 


-- Stats after removing the forwarded records. 
Scan count 1, logical reads 481, physical reads 9, read-ahead reads 9, lob logical reads Ø, lob physical reads 9, lob read-ahead reads Ø 
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The final step is to create a job that executes the 
FixForwardedRecs stored procedure during a main- 
tenance period. Although we are using the new sys 
.dm_db_index_physical_stats DMV, it still causes some 
blocking and should be 


used duringa time of low “The procedure corrects 


to no use. 


After FixForwarded forwarded records 
Recs has run, you can without permanently 
changing the table 
structure and without 
unloading and reloading 
which returns the results data in a table. 


view the logged results 
using the command 


SELECT * FROM FR_Log 


shown in Table 2, page 

27. You can see that FR 

Table started with 26,885 forwarded records, but after 
the FixForwardedRecs stored procedure was run 
against it, the table shows 0 forwarded records. 

Let’s now look at the benefits of removing for- 
warded records. Web Listing 3 shows a simple example 
of creating a heap table with a couple of non-clustered 
indexes. For the purpose of this article, I used SQL 
Server Integration Services (SSIS) to import data 
from AdventureWorks.Person.Address to populate 
the AddressHeap table with 19,614 rows. The Address- 
Line2 column has mostly NULL values in it, so I used 
the REPLICATE function to populate the Address- 
Line2 column with a string containing 20 characters, 
as shown in the following command: 


update [AddressHeap] set addressline2 = 
REPLICATE(N’x’ , 20) 


Then I updated the AddressLine2 column with a 
string containing 40 characters, as shown in the 
command 


update [AddressHeap] set addressline2 = 
REPLICATE(N’x’ , 4) 


There are now more than 8,574 forwarded records 
on the AddressHeap 
table. Now you can run 
the query in Listing 3 and 
capture the STATISTICS 
IO. Next, run the Fix 
ForwardedRecs stored 
procedure on the FRTest 
database and removed all 
forwarded records from 
the AddressHeap table. 
Then you can capture 
the STATISTICS IO 
using Listing 4. You'll 
notice there’s a 97 percent decrease in reads on a very 
simple query on a relatively small table. 


Improve SQL Server 
Performance 
The script I’ve provided will automatically remove 
forwarded records by creating a clustered index on 
either an existing auto incrementing column or a new 
dummy column that’s created as part of the process. 
In both instances, the clustered index is removed to 
return the table to its original state. You can easily 
identify heap tables that might need to be redesigned 
to include a permanent clustered index by querying 
for those tables that repeatedly appear in the FRLog 
table. Because forwarded records are a type of frag- 
mentation, resolving them should be part of your 
typical database maintenance routines. By doing so, 
you should see a positive return on performance— 
and you might even enjoy a little praise from your 
users. SQL 
InstantDoc ID 101572 
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7 Steps for Successful 


Data Warehouse 
Projects 


Project management for DBAs, developers, 


and IT pros 


uilding a data warehouse isn’t a simple task, 
R and it shouldn't be done by only one person. 

Because a data warehouse combines the best 
of business practices and information systems tech- 
nology, it requires the cooperation of both business 
and IT decision makers, continuously coordinating to 
align all the needs, requirements, tasks, and deliver- 
ables of a successful data warehouse implementation. 

Td like to share the approach I use when plan- 
ning and managing any database project, including 
transactional databases, data warehouses, and hybrid 
databases. I live in the world of relational databases, 
data warehouses, and the extraction, tranformation, 
and loading (ETL) processes that support them, so 
Tll focus my approach in this realm. However, you 
can extend this approach to the entire stack, including 
OLAP cubes and information delivery applications 
such as reports, ad-hoc analysis, scorecards, and 
dashboards. 

Tm not presuming how to tell a bona fide project 
manager (PM) how to do his or her job. Rather, I wrote 
this article for DBAs and developers who don’t have 
the good fortune to be working with an experienced 
PM, or for those IT pros who have been ordered to 
“build a data warehouse” and act as their own PM. 
My discussion won’t be complete, but I hope that it'll 
give you enough information to start the project ball 
rolling. 

A data warehouse project has three tracks—the 
Data Track, the Technology Track, and the Application 
Layer Track, as illustrated in Figure 1, page 30. When 
youre putting together any database project | plan, I 
recommend using these three tracks as a template to 
manage and synchronize your activities. You can also 
use Figure 1 as a high-level overview when explaining 
the plan to technical decision makers (TDMs), business 
decision makers (BDMs), and all other participants in 
the data warehouse project. 
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Using a Lifecyle Management 
Method 

I encourage you to take advantage of the resources 
that your organization offers, such as techniques and 
methodologies for designing, developing, and deploying 
systems and software. If your company hasn’t adopted 
any formalized methodology for doing this work, then 
go ahead and use the technique that I’ve developed for 
my own database projects, namely, my trademarked 
7D Database Lifecycle Management Method, more 
commonly known as the (also trademarked) “7D 
Method.” 

My 7D Database Lifecycle Management Method 
addresses lifecycle management of the database, not 
the lifecycle of hardware or software (applications) that 
touch the database. I’ve included both hardware and 
software tracks in Figure 1, but I won’t be expanding 
on the management of either. It’s necessary to align and 
synchronize milestones in the database lifecycle with 
both hardware and applications in order to successfully 
implement the database lifecycle methodology. 

You never truly finish building a data warehouse. 
Unlike a traditional database, which often remains 
relatively static for some extended period of time 
after deployment, a data warehouse is constantly in 
a state of flux, responding to changes in the business 
conditions for which it was created. Today’s business 
environment is more complex and deals with a faster 
rate of change than ever before. Managing nearly 
constant change is one of the greatest challenges 
of the enterprise. That’s why it’s so important that 
everyone on the data warehouse team, BDMs and 
TDMs alike, be on the same book and page, using the 
same type of lifecycle approach so that they’re totally 
aligned in their thinking. Only by doing this is there 
any hope of aligning the implemented data warehouse 
with the vision and purpose of the enterprise. In 
Figure | I’ve laid out the seven steps of my 7D Method, 
and I'll walk you through each step in this article. 


Michelle A. 
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(MAPoolet @ MountVernonDataSystems.com) is 
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z i 7 STEPS FOR SUCCESSFUL DATA WAREHOUSE PROJECTS 


Step I: Discover 
I guarantee that any database project of any appre- 
ciable size and scope will fail if you don’t start with a 
Discover stage. Also known as “requirements analysis 
and definition,” the Discover step requires a business- 
=] centric approach, especially in data warehousing 
= projects, since the output from a data warehouse needs 

| to support the organizational goals. The Discover 
step is essentially an investigation, and you should 
be constantly asking six basic questions (what, how, 
where, who, when, and why), recording the answers, 
and incorporating these answers into the solutions 
| you craft. 

In the first three steps (Discover, Design, and 

Develop) there must be concentrated coordination 
between the business owners and the technology 
; specialists; the PM should enable this process. As an 
V— Fe independent professional who’s primarily concerned 
that the project stays on time, within budget, and 
g works as promised, the PM is in charge of establishing 
è k $ critical paths, milestones, and success metrics, after 
§ 
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getting feedback from all parties. If there’s no PM on 
wens!) the project, these will be your tasks. 
In the Discover stage, the PM has to collect infor- 
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mation for all three tracks shown in Figure 1: the 
Technology Track, the Data Track, and the Applica- 
tion Layer Track. Among other tasks, the PM has 
to specify stakeholders and users, and must under- 
stand each of their roles and their data visualization 
needs. The PM has to be aware of the organiza- 
tion’s performance management strategy: What are 
the objectives, initiatives, and supporting metrics/ 
KPIs used to track the health of the business and 
the project? If any of these portions of the strategy 
are missing, then there’s a high probability that the 
project will miss its mark with end users, which would 
— result in low adoption rates and no future funding. 
In other words, the project would fail, no matter how 
flawlessly the project tasks were executed. 
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Step 2: Design 
The major activity of the Design step is the develop- 
ment of the semantic and schematic models that 
describe the data warehouse. These models need to 
address business users’ management information sys- 
— tems (MISs) and business intelligence (BI) analytical 
i needs. For a data warehouse project you can create 
j conceptual and logical data models for the relational 
data warehouse and dimensional models to represent 
— the multidimensional cubes. You can use a decision 
Figure 1 | — matrix to help determine what needs to be included 
in each dimensional model; list key business processes 
The data | f that will be supported by the data warehouse along the 
uo Y axis and the proposed dimensions along the X axis. 
management This matrix will serve as a guide for current develop- 
lifecycle ment and for future extension and cross-organizational 
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integration. The models you create for the Design stage 
have to reflect the answers to the six interrogatives you 
collected in the Discover step. 

It’s a good idea to identify all the data sources 
(internal and external), operational/transactional 
databases, and flat file structures that will be part of 
the data warehouse. You should also specify which data 
will be imported into the data warehouse and which 
will simply be referenced as an external data source. 

Typically, the Technology Track will have its own 
PM, but you might need to fill that role, too. A data 
warehouse can grow to be very large in content and 
very broad in scope, so it’s necessary to properly size 
a data warehouse before its deployment. Estimate the 
size on paper first, so you can get an idea of how much 
processing power and disk capacity you'll need when 
you take the data warehouse into production. You'll 
need to estimate the number of end users on day one 
and what kind of applications they'll be using (e.g., to 
do ad-hoc analysis against cubes or pull cached reports 
from the relational data warehouse), and the amount 
of data that will be stored in year one. You'll need a 
two-year and a five-year projection, because just as a 
data warehouse is a work in progress, so too will its pro- 
cessing and data storage needs continuously increase 
over time. Data warehouse installations include a 
variety of hardware, communication, and software 
solutions, all of which must work together to deliver 
a working data warehouse to end users. You'll want 
plenty of time to plan and test how you'll integrate all 
these different components. 

Like the Technology Track, the Application Layer 
Track might have its own PM or a lead software devel- 
oper who fills that role. If you're that fortunate, your 
job will be to coordinate with the PM to synchronize 
tasks. If not, your job description just expanded. The 
application layer includes output gleaned from the 
data warehouse, which is generally MIS reports and 
BI analysis results. MIS reports are the screen dis- 
plays, dashboards, and printed copy that help business 
managers make the tactical decisions needed to run the 
day-to-day business. These outputs are relatively easy 
to define, codify, and capture into a set of standardized 
processes that can be run in a scheduled environment. 
The BI portion of the application layer is the set of 
queries and responses that help executive management 
make the strategic decisions that drive a business. The 
BI solutions are often unstructured and hard to pre- 
define because they tend to explore the data in an 
ad-hoc manner. Scoreboards, graphs, and pivot tables 
are examples of BI applications that stimulate addi- 
tional data investigation, which can result in strategic 
directional shifts within a company. 

At this stage many methodologies call for a pro- 
totype or pilot project. The 7D Method does not. At 
most, as part of the design activities for the application 
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layer, you can do a “clickable model”—a quickly- 
rendered mockup of an input/output screen that 
involves little to no code, but gives the stakeholders a 
visualization of the concept without eating up precious 
time and resources. If a pilot or prototype is necessary, 
then choose a slice of the whole and walk the pilot 
through the steps of the 7D Method. The 7D Method 
doesn’t distinguish between pilots, prototypes, and pro- 
duction systems—they’re all treated as projects. 

If you engineer a prototype through the 7D Method 
and it ends up in production (as many prototypes do), 
then you'll have to choose the second slice even more 
carefully than the first. If the slices don’t successfully 
integrate with each other and don’t support the enter- 
prise purpose and intent you uncovered during the 
Discover step, you'll be creating “stovepipes,’ which are 
islands of information that integrate with each other 
only with difficulty, and in some cases, not at all. 


Step 3: Develop 

The Data Track Develop step has two major compo- 
nents: The first involves mapping the data models to 
their equivalent physical design counterparts (essen- 
tially, blueprints for the relational data warehouses 
and the OLAP cubes), sizing the database and parti- 
tioning the tables where necessary, setting up naming 
conventions for data warehouse objects that both 
business users and technology users can live with, and 
developing strategies for indexing and identifying index 
candidates. The second component involves developing 
schemes for the ETL of data from external sources into 
the data warehouse. Included in, but not limited, to this 
portion of the project is DTS/ SQL Server Integration 
Services (SSIS) package development and testing, 
import/export and T-SQL script development and 
testing, and data integration testing for those external 
data source components that won’t be imported into 
the data warehouse. 

The Technology Track Develop step includes 
reviewing, testing, and selecting products that will 
deliver their piece of the system architecture design. 
This has to be done for the various layers that make up 
the communications links—the physical layer; the data 
link and network layers; and the transport, session, and 
presentation layers. While many products seamlessly 
fold multiple layers into a solution, it’s necessary to be 
aware of future load requirements and performance 
demands on each of these layers in turn, and to pre- 
pare for these demands in advance. You should select 
the data warehouse servers and storage solutions, 
and new end-user-facing hardware that you'll need to 
deliver data from the new data warehouse. You'll do 
this for the production data warehouse and for the 
staging database—the place where the DTS/SSIS pack- 
ages and T-SQL scripts will be executing, importing 
data from external data sources, and exporting the 
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manipulated and massaged data into the relational 
data warehouses and OLAP cubes. Depending on the 
requirements gathered during the Discover stage, you 
may also be supporting data marts, snapshots, and 
reporting databases as part of your data warehouse 
environment, so be prepared to plan the environments 
for these, also. 

The Application Layer Track Develop step sounds 
simple: Just develop the end-user applications. However, 
this might be the most complex and time-consuming 
task in the entire process, and could be the most costly if 
success metrics aren’t carefully developed and respected. 
It’s in this stage that scope creep (i.e. continuously 
adding features and functionality without regard to 
the impact on design and development of the other 
two tracks) can torpedo the project. In addition to 
developing the end-user applications, you'll also have to 
develop a scheme for testing these applications. You'll 
need to create end-user training programs on how to 


With your guidance and shepherding, the three 
tracks will arrive at the Deploy stage when they’re 
scheduled, avoiding weeks or months of “running 
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behind schedule” angst. 


use these applications. At every milestone you'll have 
to ensure that you get sign-off from the relevant parties 
involved. 

It’s surprising how many development projects are 
staged and tested in a production environment. Don’t 
do this! Create a separate physical environment for 
developing, testing, and staging the components. You 
create a separate environment for your line of business 
(transactional) systems—do the same for your data 
warehouse. 


Step 4: Deploy 
You don’t deploy a data warehouse the same way you 
deploy a transactional database. Typically, you deploy 
a transactional database in a swift, all-encompassing 
fashion. For example, on Friday evening, end users 
are in the legacy system, and Monday morning they're 
logging into the new database. Data warehouses are 
usually deployed incrementally to various groups of 
users throughout the enterprise. The pace at which this 
happens, and the order in which the data warehouse 
is made available to individual groups, is part of the 
rollout plan contained within the Deploy stage. 
Ideally, data warehouse deployment happens in 
rapidly-cascading tiers. First the technology (eg., 
servers, storage, communication links) is put into 
place. Systems software is then installed, tested, and 
made ready for production. Then the Data Track 


components are rolled out—the data warehouse data- 
bases (relational and OLAP) are built, and the ETL 
processes are brought online. There’s usually a break 
in time before the final application layers are added, 
while you let the data flow from the external sources, 
through the ETL processes, and into the various data 
warehouse databases and cubes, testing and adjusting 
as needed. Then the application layer is deployed. 
You'll probably want to plan to deploy the application 
layer gradually, because different audiences within the 
enterprise adopt at different rates. 

As PM, you play a vital role. With your guidance 
and shepherding, the three tracks will arrive at the 
Deploy stage when they’re scheduled, avoiding weeks 
or months of “running behind schedule” angst. As 
soon as the Technology and Data Tracks are tested 
and ready to go, start rolling out the application 
layer. A data warehouse without a UI is of no use to 
anyone, and a data warehouse with an undersized and 
weakly-engineered systems architecture will perform 
so badly that it will never be adopted by the business 
users. 


Step 5: Day to Day 

Managing day-to-day operations is very important; 
this management is often overlooked in the planning 
and development process. You have to ensure that 
regular (e.g., daily, weekly) maintenance is performed 
at both the hardware and software levels, and you have 
to continuously monitor the performance and growth 
of all systems. As I said at the beginning of this article, 
a data warehouse is never finished; it will continue to 
grow and expand as more users discover the value 
inherent in the data and create new, and sometimes 
challenging, ways to query the data warehouse. Some 
of the PM tasks you must be prepared to undertake 
include ensuring that all systems (hardware, commu- 
nications links, systems software) are fully operational 
and up-to-date on patches and upgrades; when opera- 
tional bottlenecks occur diagnosing and resolving 
them as quickly as possible; ensuring that all systems 
needing to be backed up do, in fact, have backup jobs 
defined and scheduled; and that all backups are tested 
by subsequently restoring to test, development, or 
reporting databases. 

Businesses don’t stand still. They must constantly 
reinvent themselves in order to stay competitive. It’s the 
job of the data warehouse administrator to track data 
use, to gauge the importance of the data, and to detect 
when business needs start to shift. As the business 
model changes, so will the need for newer, better, more 
flexible, and possibly more complex user applications, 
and you should be aware of these requirements. In time 
there will be enough change in the direction and focus 
of the business to initiate a redo of the Discover stage, 
and the lifecycle will have come full circle. 
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Step 6: Defend 

Defending your data warehouse involves more than 
just taking regular backups or making sure that none 
of the applications include SQL queries that might be 
open to SQL injection attacks. You have to plan for 
the entire scope and breadth of defense because the 
data warehouse contains the enterprise’s most valuable 
asset—its data—in a compiled, cleansed, and (in some 
cases) informational format. 

Threats to the data warehouse usually fall into two 
categories: physical and logical. Physical threats can 
be external (e.g., tornado, flood, fire, earthquake) or 
internal (e.g., intentional, accidental). You can protect 
against physical threats by employing practices as 
simple as restricting access to the computer and com- 
munications rooms, or as complicated (and expensive) 
as mirrored servers at a failover site that’s geographi- 
cally distant. The steps you take for physical defense 
depend on your recovery time and recovery point 
objectives, that is, how long you can afford to have your 
data warehouse offline, and how much data you afford 
to lose, respectively. 

Logical threats are more complicated, simply by 
the nature of the data warehouse environment. The 
OS might fail, the database management system might 
crash, one or more applications might intentionally or 
accidentally corrupt, destroy, or misinterpret data (this 
is especially true in the ETL processes that feed the 
data warehouse). Browser UIs have exposed embedded 
SQL calls to the threat of SQL injection attacks. Each 
potential threat has to be identified and addressed; it’s 
far better to develop remedies for threats before they 
occur, rather than after they happen. 

Your job as PM is to develop a comprehensive 
defense for your entire data warehouse installation. If 
you're lucky enough to have a security administrator, 
leverage this person’s expertise and experience. 


Step 7: Decommission 

There may come a day when the data warehouse, 
or a component thereof (e.g., a staging database, a 
data mart, a reporting database, a cube), no longer 
meets requirements and needs to be decommissioned. 
Not every database can be constantly refactored or 
upgraded to meet new requirements. Sometimes you 
simply have to scrap and rebuild, especially if the 
database instance was “built on spec,” that is, it wasn't 
properly architected to fully reflect the goals and intent 
of the enterprise. When this happens, as PM, you have 
to synchronize the process. 

Typically, the Decommission step happens in one 
of three ways: decommission with no replacement, 
decommission with cutover, and decommission with 
phase in/phase out. Decommission with no replace- 
ment means that the function the database used to 
perform is no longer needed. The database is retired, 
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as well as the functions it used to perform. Decommis- 
sion with cutover indicates that another database will 
replace the one being retired, and that the functions 
performed will be quickly transferred from the old 
database to the new database. One day the users will 
be pointing to the old database, the next day they'll 
be pointing to the new one. Decommission with phase 
in/phase out indicates that the old and new databases 
will be run side-by-side for some period of time while 
functions and users are incrementally transferred 
from the old onto the new, until at last there are no 
users or functions running on the old database and it 
can be retired. Each scheme has its risks and rewards; 
as PM you have to determine when the risks outweigh 
the rewards and which scheme makes most sense for 
your situation. Then you have to plan and execute, 
working with others in the Technology Track and 
the Application Layer Track to ensure a seamless 
conversion. 


The Virtuous Cycle 

As you work with these data warehouse components 
there will be subsequent rounds of discovery, during 
which you'll assess the new requirements that have 
evolved over time. These new requirements might lead 
to enhanced designs and expanded solutions for one 
or more tracks. You'll need to integrate the changes 
in the existing data warehouse so you can deploy the 
newer, better solutions to the business users eager to 
take advantage of them. Some new requirements might 
cause changes in the day-to-day operations in order 


Even without a formal PM, even if you're 


the only person listed under “resource, human” 
on this project, you should still create a PM plan 


and keep it up-to-date. 


to keep the data warehouse running like a well-oiled 
machine. 

Over time, multiple iterations of this lifecycle 
process will ingrain the data warehouse into the fabric 
of the enterprise, until the data warehouse and the 
business become a seamless unit. The PM ensures 
that all activities and tasks are done according to 
specification, are accepted according to established 
success metrics, and are deployed synchronously. 
Even without a formal PM, even if you're the only 
person listed under “resource, human” on this project, 
you should still create a PM plan and keep it up-to- 
date. Then, when management asks “How’s the data 
warehouse project going?” you can show them exactly 
what stage the project is at and what’s been done. 

SOU 


InstantDoc ID 101562 
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Join Us at 


Tech-Ed 
2009 


TechEd is one of the most 
significant IT conferences of 
the year, and TechEd 2009 is 

no exception. Windows IT Pro 
and SQL Server Magazine will 
have a significant presence 
at the show. Here are some 
highlights to watch for: 


fence activities, products and services 


Win a Prize at the SQL Server Magazine Booth! 

Be sure to stop by the Windows IT Pro/SQL Server Magazine booth (#411) to sign up for one 
of our free giveaways and to chat with some of our editors and authors. We always like to hear 
feedback from readers, so let us know what you like (and don’t like) about our coverage. 


Author Sessions and Roundtables 
A few of our authors are giving presentations at the show. SQL Server Magazine 
Contributing Editor Kalen Delaney is speaking about solving real-world DBA 
issues. SQL Server Magazine Contributing Editor Itzik Ben-Gan 
will be speaking on T-SQL. Windows IT Pro Senior Contributing 
Editor Mark Minasi will deliver sessions on Windows Server 2008 
R2 AD features, Windows Kerberos, and Security with UAC/WIL. 


2009 Best of TechEd Attendees’ Pick Awards 

You'll also want to cast your vote in the 2009 Best of TechEd Attendees’ Pick Awards, which 
lets TechEd attendees pick their favorite products on display at the show. You'll 
find voting kiosks scattered throughout the show floor that you can use to log 
on to the contest website and cast your ballot. 


Live Blogs and Twitter Feeds 


We'll be covering the show with some live blogs and Twitter feeds, so be sure to bookmark the 


WindowsITPro.com and SQLMag.com websites and follow our Twitter accounts for updates: 


E Best of TechEd Awards: www.twitter.com/bestofteched09 


E Windows IT Pro: www.twitter.com/Windowsitpro 
E SQL Server Magazine: www.twitter.com/SQLServerMag 


E Jeff James: www.twitter.com/jeffjames3 


Amy Eisenberg: www.twitter.com/witproamy 


E Sheila Molnar: www.twitter.com/sqlmagsheila 


SOL SERVER , 
Time Machine? 


SQL diagnostic manager” History Browser 
e Rewind back to the exact time a problem occurred 


e View key SQL Server performance metrics 


e Diagnose and solve problems 


e 
TRY IT FOR FREE: IDERA.COM/TIMETRAVEL iC) (E To} 


TRAVEL BACK IN TIME AT TECH-ED-VISIT US BOOTH #436 


AX )|Milze 


STORAGE PERFORMANCE 


Increase performance 
without increasing costs 


nce upon a time, a client engaged me to help 
o) justify the pending purchase of a $100,000 
SAN. With 3,000—4,000 concurrent users 
accessing over 150GB of data, the client was starting to 
encounter performance problems. Hardware vendors 
had recommended a new SAN to tame the I/O bottle- 
neck, and I was on hand to OK the proposed solution 
and help with the migration, should one be needed. 
However, an evaluation of the client’s workload 
and current environment made it painfully obvious 
that although more than $35,000 worth of disks, 
RAID controllers, and other hardware was already 
in place, it simply wasn’t being utilized efficiently. By 
using only the existing hardware, we increased per- 
formance and eliminated the “need” for an expensive 
SAN. (In hindsight, I should have charged a commis- 
sion on the money saved, instead of merely charging 


puters, and Table 3, page 36, lists common disk rota- 
tional speeds. Each table tells you how good different 
hardware choices are for SQL Server. The tables make 
it obvious that not all storage is created equal—which 
is fine, as performance requirements vary from deploy- 
ment to deployment. Accordingly, I like to approach 
disk optimization scenarios by identifying what I call 
“premium disk,” the most performant storage option 
available. For example, in the disk configurations 
shown in Figure 1, page 36, the premium disk on Server 
1 is obviously the 220GB volume, while on Server 2 it’s 
the 640GB volume. The key to identifying premium 
disk is the fact that almost all SQL Server deployments 
have access to at least two kinds of disk. 

Premium disk capabilities vary widely, but you 
can usually count on two things. First, fast premium 
disks are also small because high-performance drives 


Michael K. 
Campbell 


mike @ sqlservervideos.com) is a consultant 
with years of SQL Server DBA and develop- 


ment experience. He spends most of his time 
engaged in consulting, technical evangelism, and 


creating free online SQL Server videos. 


by the hour.) By applying the 
following strategies, which I 
used to help my client, you can 


TABLE |: Commonly Used RAID Types 


Best performance. Reads and 
writes are split evenly over 
all disks in the array, making 
them very fast. 


Reliable fault-tolerance 
because data is written to 
all disks in array. Reads can 
be split over all the disks in 
the array. 


Excellent redundancy: 

A single drive can fail without 
jeopardizing the entire array. 
Decent read performance. 
Very affordable. 


Combines strengths of 
RAID-0 and RAID-1 to deliver 


boost performance and pos- "ype Pros 
sibly save yourself significant RAID 0 
hardware costs. (striping) 
Storage Is Not 
Created Equal RAID 1 
Disk drives keep getting (mirroring) 
less expensive, but the cost 
for swarms of high-end hard 
drives and RAID controllers is 
still a major concern for most RAID 5 
organizations. Further, some (Striping with 
types of storage and hardware parity) 
are better choices for use with 
SQL Server than others. 

Table 1 lists some common seit ain 
types of RAID arrays, Table earn 


2, page 36, lists common ways 
for storage to connect to com- 
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fault protection and great 
performance. 


Cons 


No redundancy. Loss of a single 
drive results in complete data 
loss. 


Writes are slowed to the speed 
of a single disk because all 
writes are written to each disk. 


To enforce redundancy, RAID-5 
solutions require two reads 
and two writes for every logical 
write operation, making writes 
very expensive. 

Requires lots of disks and 


high-end controllers. One of the 
most expensive RAID solutions. 


Recommendation 


Not recommended for use in SQL 
Server environments due to the 
risk of data loss. 


Works well for low-demand data 
files. Excellent for log files. 


Affordable solution for high-per- 
formance workloads that don’t do 
lots of writes. Don’t use in systems 
where writes represent more than 
10 percent of overall activity. 


Excellent for all high-performance 
database needs. Slight overkill for 
log files in some cases. 
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TABLE 2: Common I/O Buses 


Bus Speed 

Serial ATA Up to 300Mbps! 
(SATA) 

SCSI Up to 320Mbps? 


Serial Attached Up to 375Mbps 
SCSI 

iSCSI Up to 125Mbps? 
Fibre Channel Up to 425Mbps* 


For comparison purposes, USB is 60Mbps and FireWire 800 is almost 100Mbps. !SATA 300; SATA 150 is 
only half of a SATA 300 bus and doesn’t support multipath 1/0. 2with Ultra320 SCSI; other types of SCSI 


Summary 


Inexpensive successor to ATA. Easy to manage and configure. 
Currently there are no 15,000rpm SATA disks available and 
only a few 10,000rpm SATA disks. 


Historically high performers. Slightly difficult to configure. More 


expensive than SATA but have best support for high-speed 


(10,000 and 15,000rpm) disks. Supports multiple devices per 


channel. 


Successor to SCSI (despite slightly lower speeds). Easier to 
configure than SCSI but still maintains access to high-per- 
formance disk drives. Provides higher redundancy than SATA 


controllers and can use high-speed SCSI disks or cheaper SATA 


disks if desired. Marketed at enterprise servers and widely 
used today. 


Commonly used in SANs, where SCS! commands are relayed 
over Ethernet. Low throughput is acceptable for backups and 


other low-level needs, but not a good candidate for highly used 


SQL Server resources. 
Lots of configuration options exist; can be used to connect to 


both internal and external drives. Commonly used in clustering 
scenarios against high-speed SCSI disks for high-performance 


and high-availability solutions. Extremely fast, but expensive. 


buses also exist. ‘over Gigabit Ethernet. ‘with 4GBps Fibre Channel 


TABLE 3: Disk Rotational Speeds 


Rotation Speed Rough Seek Time 


5,400rpm 5+ ms 

7,200rpm 4 ms 

10,000rpm 3 ms 

15,000rpm 2 ms 
36 April 2009 


Summary 
Low speed, low heat, low performance. Common in laptops. 


Suitable for servers where disk activity is minimal. Common 
in desktops. 


Capable of 125Mbps throughput. 
Trades capacity for quickest access speeds and throughput. 


generally sacrifice capacity for speed. Second, 
there’s less premium disk available than non- 
premium disk and, if the system has been 
around for a while, there’s less premium disk 
capacity available than you want. The key to 
striking a balance between performance and 
cost lies in optimizing the performance of 
premium disk while off-loading as many tasks 
as possible to non-premium disk. 


Multiple Data Files 
A key aspect of optimizing premium-disk per- 
formance is to adequately harness and utilize 
controllers and disks. Resource contention 
can result in hotspots where storage require- 
ments aren’t evenly spread across all disks, so 
avoiding contention is one way to make sure 
that your storage systems are properly utilized. 
For large, heavily used databases, Microsoft 
recommends that you use between 0.25 and 
1 data file per file group for each CPU on a 
server. With this approach, SQL Server can 
spawn multiple threads to satisfy I/O require- 
ments simultaneously for queries that it can 
process in parallel instead of forcing query 
execution to wait on a single thread to pull 
data from disk. Even with expensive RAID 
controllers and 15,000rpm disks, if you have 
only one spindle or disk handling your work- 
load, you could be missing out on perfor- 
mance benefits that SQL Server can provide 
by efficiently using multiple I/O threads to 
satisfy large and complex queries. Even if you 
have plenty of high-performance disk space, 
disk is much slower than RAM or CPU, so it 
has the greatest potential to destabilize your 
workloads if it isn't being used efficiently. 
Merely adding multiple data files to large, 
heavily used databases can help SQL Server 
achieve greater parallel execution and shorten 


Server 1 
8 cores; 64GB RAM 


Server 2 
2 cores; 32GB RAM 


220GB RAID 1+8 with 15,88@rpm Fibre Channel-attached drives 
1TB SAN (RAID 5 on 18,8@@rpm iSCSI-attached drives) 


64ØGB RAID 5 on 72øØØRPM SATA disks 
8Q@GB RAID 1 accessed via Network File Share 


Figure | 


Sample SQL Server configurations 
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backup times, but intelli- 
gently distributing data over 
multiple files will lead to the 
best results. For example, if 
you have a large table that’s 
frequently targeted by many 
ranged queries, spreading 
that table across several files 
can allow multiple sequen- 
tial reads to satisfy query 
needs instead of forcing 
execution to pull everything 
from a single file. The effect 
of splitting files is more pro- 
nounced when you're able to 
place these files on different 
disk arrays. Likewise, it’s 
usually also a good idea to 
split the indexes for large, 
heavily used tables into a 
dedicated file group. Split- 
ting the indexes this way 
allows parallel processing of 
index and table data in more 
complicated queries that 
require lots of filters and bookmarking operations. 


“Fill Factor,” 


Log File Placement 

Unlike data files, log files don’t benefit from being 
spread across multiple files because of the sequential 
way SQL Server writes log file data. But you should 
place log files on dedicated disk arrays when possible 
because they can generate large amounts of I/O when 
many updates are being made. Using dedicated disk 
arrays helps isolate log-file write activity from the I/O 
activity of queries and data modifications in the rest 
of the database. RAID 5 arrays require two reads 
and two physical writes for every logical write, so you 
should generally avoid using RAID 5 arrays for log 
files. RAID 1+0 arrays are the best option, but RAID 1 
arrays with fast disks can handle very heavy workloads 
when used exclusively for log files. 


Mind Fragmentation 

and Fill Factor 

On large tables and indexes that see lots of updates 
and deletes, fragmentation can easily result in wasted 
space that can hurt I/O operations as SQL Server 
pages table and index data into memory. Often, frag- 
mentation can get so bad that it results in a noticeable 
lag in response time for end users on heavily queried 
tables. Many experts, including some at Microsoft, 
recommend regularly rebuilding and defragmenting 
indexes and fine-tuning the fill-factor setting using the 
CREATE INDEX statement or the ALTER INDEX 
statement to combat excessive fragmentation and lag. 
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LEARNING PATH 
SQL SERVER MAGAZINE RESOURCES 


To learn more about SQL Server storage: 

“Other SQL Server Storage Options,” InstantDoc ID 98712 
“What’s the Best Way to Carve Up a SAN?” InstantDoc ID 96555 
“Selecting a Storage Array for a SAN,” InstantDoc ID 48485 


To learn more about SQL Server performance: 
“Database Design for Performance,” InstantDoc ID 93633 


“Under the Hood: Performance Tuning SQL Server 2008 (video),” 
InstantDoc ID 98036 


“Performance Secrets for SQL Server Developers,” InstantDoc ID 99148 


“How to Improve Application and Database Performance up to 40% in 
One Easy Step,” InstantDoc ID 100503 


To learn more about fill factor: 
“Sharpen Your Basic SQL Server Skills,” InstantDoc ID 97337 


MICROSOFT RESOURCES 
“Storage Top 10 Best Practices,” 


http:/Avww.microsoft.com/technet/prodtechnol/sq|/bestpractice/ 
storage-top-10.mspx 


http://msdn.microsoft.com/en-us/library/ms177459.aspx 


(See the Learning Path for more information about the 
fill-factor setting.) 

Fine-tuning the fill-factor setting on heavily queried 
and written tables can be difficult. Too much fragmen- 
tation results in wasted read operations, and too little 
free space can significantly affect insert speeds. There’s 
no one-size-fits-all approach, nor even a reliable rule 
of thumb: The best solution takes environmental 
concerns and usage patterns into account. The best 
way to approach the fill-factor setting is to routinely 
gather and analyze performance metrics, then make 
small changes, monitor their impact, and readjust 
when necessary. This approach can take a bit of effort, 
but if you can establish optimal fill-factor settings for 
key tables and use off-peak times to rebuild your tables 
and indexes accordingly, you can dramatically boost 
performance in cases involving large tables. There’s no 
reason you can’t rebuild indexes nightly, as long as you 
have some downtime available. 


Available Free Space 

Although maximizing every ounce of premium disk 
is a worthwhile goal, be sure you keep an eye on the 
amount of free space available. If your disks have to 
work overtime to find free space on the drive, you'll 
see performance degrade dramatically. The point at 
which this saturation occurs varies from disk to disk 
and depends on a variety of factors, including the raw 
performance characteristics of the drives and their 
associated controllers, but a good rule of thumb is 


The best way 
to approach 
fill factor is 

to gather 

and analyze 
performance 
metrics, make 
small changes, 
monitor their 
impact, and 
readjust when 
necessary. 
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ivi 


Updates / Act 


07:00 


12:00 18:00 


Time of Day 


in case of disaster. The last thing 
you want in an emergency is 
to wait while your backups are 
sucked over the wire on a 100 
megabit or gigabit network. 


Efficient Log File 
Management 

Many SQL Server environments 
have only one RAID controller 
or one RAID array, which data 
and log files must share. Conse- 
quently, log files end up on pre- 
mium disk and can devour lots 
of high-performance disk that 
they don’t need. For example, 
if you have a 100GB database 


23:59 


Figure 2 


Usage versus time of 
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day 


to maintain at least 15 to 25 percent of free space on 
your drives. 

One great way to free up premium disk is to off- 
load all but the most critical databases to less expensive 
storage. By putting such databases on a SAN or other 
inexpensive storage, you can free up premium disk 
for mission-critical databases and operations. This 
is especially true for small shops whose staging and 
development databases commonly reside on the same 
hardware as production databases. 

Partitioning can be a powerful tool for off-loading 
data, and SQL Server 2005’s partitioning schemes 
make off-loading data a breeze. Just create your file 
groups and add your files, putting your less frequently 
used files on cheaper disk. Create your partition 
scheme, and SQL Server will do the rest. 


intelligently Manage Backups 
Backups are a double-edged sword. Ideally, you want to 
back up large databases to premium disk to maximize 
throughput. The problem is that after your backups are 
finished, the only thing they require is lots of disk—and 
they can gobble up premium disk in a hurry. Because 
most full backups are done during off-peak hours, I typi- 
cally recommend putting backups on non-premium disk 
when there isn’t enough premium disk space available. 
This approach allows backups to take up space only on 
non-premium disks, and when done during off-peak 
hours the performance overhead associated with using 
non-premium disk doesn’t have a big impact. 

Another option for backups is a third-party com- 
pression backup agent. These are typically much cheaper 
than additional premium disk and provide the added 
benefit of decreasing recovery times. Off-box backups to 
an iSCSI SAN or a network file share can also be good 
options, but a good rule of thumb is to always have one 
or two days of complete backups, including full backups 
and logs, in easily accessible storage such as on the box 


that sees about 12GB of logged 
activity each day, you likely want a log file of about 
25GB, enough to handle daily load and account for 
spikes in activity. This, in turn would consume 25GB of 
premium disk, but log file information is perfectly safe 
on lower-end storage, such as a RAID 1 array. The net 
result is that you can end up wasting lots of premium 
disk if you're only backing up your log files on a daily 
basis—and backing up only once a day is a bad move 
from a recoverability standpoint anyhow. 

Because log file backups and the associated trunca- 
tion operation are extremely performant, using regular 
and frequent log-file backups can help maximize 
the amount of premium disk capacity available. For 
example, consider the typical workday schedule in 
Figure 2, in which most updates and modifications 
happen during peak hours. In the case of the 100GB 
database mentioned earlier, implementing hourly 
backups of the log allows the log file to be only 4GB, 
a comfortable size, assuming roughly 1GB-2GB 
of modifications per hour during peak hours plus 
ample padding. Hourly backups allow completed 
transactions to be safely logged and backed up to non- 
premium disk, freeing more than 20GB of premium 
disk without sacrificing reliability or performance. 
On larger systems that are slowly growing over time, 
this approach, properly used, can reduce the need to 
purchase additional premium storage. 


More Performance, Less Money 
Ultimately, managing disk performance calls for effort, 
tuning, tweaking, and even educated guesses. It’s worth 
the effort, however: By using your existing resources 
creatively, you can maximize the efficiency of your 
expensive I/O subsystems, which in turn can boost 
SQL Server's overall performance and save the money 
you'd otherwise spend to upgrade to a more expensive 
storage solution or purchase new storage. SQL] 
InstantDoc ID 100893 
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T-SQL IOI: 


The CASE Function 


In Lesson 9, learn how to transform data 
with two types of CASE functions 


ata is often stored in a format that’s 
Don for speed and efficiency but not 
necessarily understandability. Making sense of 

long lists of numerical codes might be second nature for 
some, but most people prefer to have more meaningful 
information displayed. Using T-SQL’s CASE function, 
you can quickly transform data from one format to 
another. Before 

ORE on the WEB 1 tell you about 

Download the code at 


the two types of 
InstantDoc ID 100152. CASE functions 


and give exam- 
ples of how to use them, you need to make sure you have 
the right tables to run the sample code. 


The Prerequisites 

To run the sample code in this lesson, make sure your 

MyDB database contains the following tables: 

e The Employee table created in Lesson 3 

e The Movie table created in Lesson 5 

* The revised MovieReview table created in Lesson 6 
(and not the original MovieReview table created in 
Lesson 4) 


If you haven’t created these database objects, you'll 
find the code in the 100152.zip file. To download 
this file, go to www.sqlmag.com, enter 100152 in 


the InstantDoc ID text box, and click the 100152 


.zip hotlink. 


The Two Types of 

CASE Functions 

The CASE function is a powerful tool for evaluating 
several conditions and returning a single value for the 
first condition met. This function comes in two types: 
simple and searched. The simple CASE function 
examines an expression and compares it to a list of 
expressions. If a match is found, a specified result is 
returned. The searched CASE function examines a set 
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of Boolean expressions. If a Boolean expression evalu- 
ates to true, a specified result is returned. 
The simple CASE function consists of the fol- 

lowing components: 

* The CASE keyword 

° The input expression to be evaluated 

e One or more WHEN clauses that specify a com- 
parison expression 

e For every WHEN clause, one THEN clause that 
specifies the expression to be returned when a match 
is found 

e An optional ELSE clause that specifies the 
expression to be returned when no matches 
are found 

* The END keyword 


The searched CASE function consists of the fol- 

lowing components: 

* The CASE keyword 

e One or more WHEN clauses specifying a Boolean 
expression to be evaluated 

e For every WHEN clause, one THEN clause that 
specifies the expression to be returned when the 
Boolean expression evaluates to true 

e An optional ELSE clause that specifies the expression 
to be returned when no 
Boolean expressions 
evaluate to true 

* The END keyword 
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Bill McEvoy 


bill @ cookingwithsql.com) is Master Chef/DBA 
for the Cooking with SQL website. He's been a 


DBA since SQL Server 4.2, and he specializes 
in batch processing and performance tuning. 


To read the previous T-SQL 101 lessons, go to 


When you're writing 
a CASE function, you 
need to pay special atten- 
tion to the ordering of the 
WHEN clauses. CASE 
functions evaluate WHEN 
clauses in sequential order 
and terminate on the first 
match found. 


“T-SQL 101, Lesson 1,” 
“T-SQL 101, Lesson 2,” 
“T-SQL 101, Lesson 3,” 
“T-SQL 101, Lesson 4,” 
“T-SQL 101, Lesson 5,” 
“T-SQL 101, Lesson 6,” 
“T-SQL 101, Lesson 7,” 
“T-SQL 101, Lesson 8,” 


InstantDoc ID 97724 
InstantDoc ID 98105 
InstantDoc ID 98315 
InstantDoc ID 98711 
InstantDoc ID 99137 
InstantDoc ID 99448 
InstantDoc ID 99765 
InstantDoc ID 99832 
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An Example of a Simple CASE 
Function 

To see a simple CASE function in action, execute the 
code in Listing 1 in the MyDB database. This query 
uses the Movie and MovieReview tables to produce a 


LISTING I: Query That Uses a Simple 
CASE Function 


SELECT m.Movie, 

mr.Stars, 

CASE (Stars) 
WHEN 1 THEN ‘Run away!' 
WHEN 2 THEN "Demand a refund' 
WHEN 3 THEN ‘Bring a pillow' 
WHEN 4 THEN 'It''s a rental' 
WHEN 5 THEN 'See it on the big screen' 
ELSE 'Not yet rated' 

END AS 'Short Review' 

FROM MovieReview mr 


@RIGHT OUTER JOIN Movie m ON m.movieID = mr.MovieID 


WHERE EmployeeID = 1 OR EmployeeID IS NULL 
ORDER BY Stars DESC 


Bridge Over The River Motherboard 
The Perfect Requirements Document 
Planet Of The Ape-Like OBA's 

The User who Knew Too Much 
Chariots of Firewire 


v for Vendor 


Bits, Bytes, Videotape 


Meet the Clusters 
Defragger Hill 


when Hery Re-Indexed Sally's Table 1 

nator: Connection Day 
The pay the Hard Drive Stood still 
a Mocking Nerd 


SCSI Term 
To Kil 


Stars Short Review 


5 See it on the big screen 
5 See it on the big screen 
5 See it on the big screen 
5 See it on the big screen 
5 See it on the big screen 
c It's a rental 

4 It's a rental 

3 Bring a pillow 

1 Run away! 

Run away! 

NULL Not yet rated 

NULL Not yet rated 

NULL Not yet rated 


Figure | 


Short movie reviews 


LISTING 2: Query That Uses a Searched CASE Function 


SELECT LEFT(FirstName + ' ' + LastName, 28) AS ‘Employee’, 


Salary, 
CASE 


WHEN (Salary BETWEEN Ø AND 2590) THEN ‘Demand justice’ 
WHEN (Salary BETWEEN 25909 AND 59090) THEN 'Ask for a raise' 
WHEN (Salary BETWEEN 58988 AND 75890) THEN 'Ask for an office' 


A WHEN (Salary BETWEEN 75988 AND 190900) THEN 'Ask for an assistant' 


WHEN (Salary >= 108900) THEN 'Pinch yourself' 
END AS ‘Career Advice' 


FROM Employee 


ORDER BY Salary ASC 


Employee Salary Career Advice 
Napolean Lawrence 23500 Demand justice 
Michael Smith 45000 Ask for a raise 

8111 Diamond 65000 Ask for an office 
John smith 66000 Ask for an office 
Garth Vader 80000 Ask for an assistant 
Raoule Teteblanche 95000 Ask for an assistant 
Garret Testerson 100000 Ask for an assistant 
william McEvoy 250000 Pinch yourself 


Figure 2 


Career advice report 
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list of short reviews for the movies seen by a particular 
employee. The query’s CASE function adds a short 
review based on the value in the Movie table’s Stars 
column. 

Note the use of the RIGHT OUTER JOIN clause 
in callout A in Listing 1. As I discussed in “T-SQL 
101, Lesson 5” (July 2008, InstantDoc ID 99137), this 
type of join causes all records in the Movie table to 
be included in the result set, even if there are no cor- 
responding records in the MovieReview table. Noncor- 
responding MovieReview records are represented by 
NULL values. Because this code is filtering the records 
to show only reviews conducted by the employee whose 
ID is 1, you have to include the code OR EmployeeID 
IS NULL in the WHERE clause. Otherwise, these 
noncorresponding MovieReview records would be 
removed from the result set. The CASE function’s 
ELSE clause transforms each NULL in the Stars 
column to the string Not yet rated in the Short Review 
column. Figure 1 shows the results. 


An Example of a Searched 
CASE Function 

To see a searched CASE function in action, execute the 
code in Listing 2 in the MyDB database. This query 
produces a “career advice” report based on the salary 
information stored in the Employee table. The CASE 
function examines the salary of each employee and 
produces a custom message depending on the range 
the salary falls within. I wrapped the salary ranges in 
parentheses for readability and to indicate that each 
range statement is a Boolean expression. Figure 2 
shows the results. 

In Figure 2, note that Garret Testerson’s salary is 
$100,000, which matches the upper range specified in 
the BETWEEN 75000 AND 100000 portion of the 
WHEN clause highlighted in callout A in Listing 2. 
Because the BETWEEN clause is inclusive (i.e., both 
the upper and lower values specified are included in 
the range), this Boolean expression evaluates to true. 
Garret’s salary also meets the condition in the last 
WHEN clause, but by this point, the CASE function 
has already found a match and has terminated. 


The Case for Using CASE 
Simple and searched CASE functions let you easily 
transform and categorize data stored in databases. As 
the code in Listing 1 and Listing 2 show, you can use 
CASE functions in SELECT statements. However, you 
aren't limited to SELECT queries. You can use a CASE 
function to transform data in an UPDATE statement, 
in an INSERT statement, or wherever a valid SQL 
Server expression is required. So, the next time you're 
faced with having to transform data, try using a CASE 
function. You'll be glad you did. SQL] 
InstantDoc ID 100152, 
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Views from the Field: 
Expert Advice on 
Virtualizing SQL Server 


Tips for planning and managing your virtual 


database environment 


irtualization has taken the computing world 

by storm over the past few years and can help 
DBAs and IT pros get the most out of their existing 
hardware resources. It’s well known that virtualization 
has tangible cost and performance benefits, including 
server consolidation, power consumption reduction, 
and the painless creation of virtual development 
and test environments. Just about everyone in IT is 
using virtualization to some degree, but many DBAs 
have concerns about running SQL Server in virtual 
environments. 

Virtualization has emerged as a valuable tool 
to help DBAs manage their database environments 
more effectively. That said, there’s a fair amount of 
half-truths and misconceptions about using SQL 
Server with virtualiza- 
tion, such as the admo- 
nition to never run a 
high-transaction SQL 
Server database in a 
virtual machine (VM) 
or that running a SQL 
Server database ina VM 
always results in slug- 
gish performance. (See 
“Virtualization Myths 
and Misconceptions” 
at www.windowsitpro 


Brent Ozar, SQL Server 
domain expert at Quest 
Software 


100790, for more infor- 
mation about this topic.) 
“We DBAs are condi- 
tioned to demand the fastest of everything—RAID 
10 drives, all the memory we can cram in the box, 
enterprise edition features, etc.,” says Brent Ozar, a 
SQL Server domain expert at Quest Software. “In 
an ideal world, we’d never put databases on virtual 
servers. But in an ideal world, all of our companies 
would be massively profitable and we’d be awash with 
help from skilled junior DBAs at our beck and call. 
This is not that ideal world, and there are cases where 
high-transaction SQL Server databases will function 
fine in virtual environments because that’s what the 
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business needs dictate.” Although virtualization does 
have some drawbacks, it can be used successfully in 
many SQL Server environments if you plan your 
virtualization strategy carefully. 


Creating a SQL Server 
Virtualization Strategy 

Because every SQL Server installation is different, 
Lindsey Allen, principal program manager lead on 
the SQL Server team at Microsoft, suggests that you 
first assess whether virtualization would be ideal in 
your environment. “[You] need to identify the goals 
the organization wants to achieve via virtualization. Is 
it cost reduction through consolidation, higher avail- 
ability, or improved manageability? Then you should 
identify the workloads running on the SQL Server 
instance to be virtualized, and test to ensure that they 
meet operational SLAs in a virtualized environment,” 
says Allen. Based on that information, you should be 
able to plan, design, and implement a new virtualiza- 
tion environment. 

“Think of virtualization like facial tissue—soft and 
disposable. You don’t want to put your production sys- 
tems onto virtualization without a lot of need driving 
it and without a lot testing to ensure it will work,” says 
Kevin Kline, director of technology for Quest Soft- 
ware’s SQL Server Solutions Group. “Instead, start 
with the low hanging fruit of database applications— 
development environments, QA environments that 
don’t do performance testing, training environments, 
that sort of thing. Only consider production environ- 
ments when you know exactly what the workload 
profile of the production application is, and you know 
exactly what the performance profile of the hardware 
plus the virtual machine(s) will be.” 

Ozar agrees, and reinforces the argument that 
up-front planning when it comes to your SQL Server 
virtualization strategy will pay significant dividends in 
the long run. “Before you start, define what ‘success’ 
means on your virtualization project. Establish service 
level agreements with the application’s business users 
before you start, and monitor your performance and 
availability in order to determine the virtualization 


Jeff James 


(james @ windowsitpro.com) is Editor-in- 
Chief, Web Content Strategist for Penton 
Media’s IT Publishing Group. He specializes in 
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stories) you'd like to 
share? Drop me an email 
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alization tips and tricks in 
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project’s success,” says Ozar. “Database administrators 
struggle with the SLA concept; we don’t always do a 
good job of defining exactly how fast an application 
needs to be in order to be considered ‘up.’ If we don’t 
agree on service level agreements before we start vir- 
tualizing, we can end up in a scenario where our end 
users complain that the apps aren’t 
fast enough, but IT believes every- 
thing’s just fine because we're saving 
so much money. [You also need 
to] establish transparency between 
the DBAs and the virtualization 
admins. The DBAs need to be able to 
trust that their virtual servers aren’t 
being unnecessarily throttled to low 
resource ceilings, and the virtualiza- 
tion admins need to trust that the 
DBAs aren't rebuilding indexes in 
the middle of a workday. This kind 
of cooperation facilitates successful 
virtualization projects.” 

Ozar also encourages DBAs to 
work closely with their colleagues in 
the IT department, mainly to encourage clear lines 
of communication and help spot potential problems 
before they occur. “Make absolutely, positively cer- 
tain that your antivirus software is not doing regularly 
scheduled scans on virtual servers,” says Ozar. “If all 
of the virtual guests kick off a virus scan on Friday 
at 9 P.M., as some antivirus software does by default, 
it will bring all of the virtual servers to their knees— 
even if they don’t have antivirus software installed. 
That’s a prime database backup time 


Kevin Kline, director of 
technology for Quest 
Software’s SQL Server 
Solutions Group 


high I/O SQL Server installations, Allen encourages 
DBAs to evaluate them on a case-by-case basis. “Vir- 
tualization adds overhead with respect to memory, 
CPU, network and disk I/O operations and there are 
inherent memory and CPU limitations for VMs,” 
Allen says. “High-transaction applications need to be 
tested to ensure they meet their opera- 
tional SLAs in virtualized production 
environments. For applications that 
require a native environment, multi- 
instance consolidation could be one 
of the solutions. SQL Server 2008 sup- 
port for hardware hot-plug (Memory/ 
CPU) is a very useful feature for such 
scenarios.” 

For more information about 
using SQL Server with Hyper-V, 
Allen suggests reading the Microsoft 
article “Running SQL Server 2008 
in a Hyper-V Environment” (tinyurl 
.com/can4g3), which provides several 
tips and suggestions for virtualizing 
SQL Server using Hyper-V, including 
recommending the use of emulated devices for SQL 
Server deployments and configuring pass-through 
disks in the guest VM root partition as offline using 
the Hyper-V Volume Manager or the DiskPart 
utility. 

Allen points out that although both SQL Server 
2008 and SQL Server 2005 can be used in vir- 
tual environments, SQL Server 2008 brings some 
extra virtualization-friendly features to the table. 

“Resource Governor provides isolation 


window, and that can wreak havoc on 
SQL Servers when they can’t get I/O 
responses back in under 15 seconds. 
Transparency between the DBA and 
system admin teams helps make sure 
this kind of global scheduling problem 
is averted.” 


Using SQL Server with 
Hyper-V 


If you're looking at teaming SQL Server Lindsey Allen, principal 
with Microsoft Hyper-V, Allen has some Program manager lead 
on the SQL Server team 
at Microsoft 


pointers to optimize VM performance 
when running SQL Server. “For the best 
possible VM performance we recom- 
mend provisioning fewer total VM logical processors 
than the total number of physical CPU cores available 
on the server,” says Allen. “You can over-provision and 
run many VMs on a single box, but the VM overhead 
([the] ratio of an operation done in a VM to the same 
operation done natively) will go up. This may not be 
acceptable in some scenarios.” 

When it comes to virtualizing high-transaction and 


and resource optimization amongst 
instances within a VM,” says Allen. 
“Data compression ensures disk space 
optimization, allowing the database 
instance and the VM to better leverage 
disk space and make better use of I/O 
bandwidth, therefore extending the 
possibility of running more VMs on 
the same server.” 

Ozar agrees that SQL Server 2008 
offers some virtualization-friendly fea- 
tures and singles out the Resource 
Governor as his favorite. “[Resource 
Governor] allows DBAs to set up per- 
formance ceilings for any given applica- 
tion or database on that server. After consolidating 
lots of applications onto less hardware, the Resource 
Governor ensures that one poorly behaved application 
doesn’t bring every other consolidated application to 
its knees,” says Ozar. “It’s the SQL Server equivalent 
of VMware resource pools: We can define exactly how 
much resources a given application gets. Even better, 
the Resource Governor can allow an application to get 
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all the horsepower available if nothing else is active at 
that particular time, too.” 


Using SQL Server with 

ESX Server and Virtual 
Infrastructure 3 

If you opt for running SQL Server with VMware’s 
ESX Server or Virtual Infrastructure 3 (VI3), many 
of the same planning and scheduling tips apply. 
According to Scott Drummonds, VMware’s group 
manager of technical marketing, many of the miscon- 
ceptions people have about virtualizing SQL Server 
have less to do with the technology and more to do 
with faulty planning and implementation. “In some 
cases, the VI3 admin may be a different person than 
the DBA and there may also be a storage admin,” says 
Drummonds. “All of these people need to work closely 
together, mainly to ensure that you don’t have a situa- 
tion where there are 10-15 databases sharing the same 
LUN [Logical Unit Number]. You can get virtualiza- 
tion performance up to and higher than 90 percent if 
the environment is properly configured.” (See “SQL 
Server Performance in a VMware Infrastructure 3 
Environment” at tinyurl.com/df6ulf for more details.) 

Obviously, the best virtualization tips and advice 
will come from someone who uses SQL Server in a 
live production environment, which is exactly what 
Tom Gibaud, manager of IT for Rochester General 
Health Systems (www.viahealth.org), does on a daily 
basis. “We’ve been using VMware for about three 
years ... we started with the low-hanging fruit of 
server consolidation and virtualizing our file/print 
servers and Active Directory,” says Gibaud. “We’ve 
been virtualizing SQL Server with VI3 for some time, 
and we’ve had great success with it.” 

Gibaud said that all but one of his organization’s 
SQL Server databases are virtualized, including 
those that handle electronic medical records, payroll, 
dictation systems, the blood bank system, radiology 
imaging, and CRM applications. More than 400 
guests inside of 60 SQL Server instances have been 
virtualized in his environment. 

Gibaud stresses that reliability is a vital part of 
his virtualization strategy. “We also looked at using 
clustering with SQL Server, but—for me, at least—it 
caused more issues than it solved,” says Gibaud. “We 
looked at VMware, and the reliability and high avail- 
ability options were very important for us. Our top 
priority is patient care, so we need to make sure that 
nurses and doctors can get the information they need 
as soon as possible at any hour of the day.” 

When asked what advice he would give other DBAs 
and IT pros when it comes to planning their SQL 
Server virtualization strategy, Gibaud suggests you 
not mix workloads in your VMs, which makes it easier 
to manage them. He also stresses the importance of 
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proper planning and running VMs on the best hard- 
ware available. “In talking to some colleagues that have 
been unsuccessful [virtualizing SQL Server], they’ve 


done things like mix their 
file and print servers with 
SQL Server in the same 
VM, or they didn’t have 
enough iron underneath 
VMware [VI3] to handle 
the workload.” Gibaud 
says that the time spent 
properly planning and 
configuring your storage 
options can also pay huge 
dividends. “You need to 
lay things out properly 
on disk. We make sure 
that our transaction logs 
are on RAID 1, data- 
bases are on RAOD S..... 
You need to follow best 
practices and set up your 


“This is not that ideal 
world, and there are 
cases where high- 
transaction SQL 
Server databases will 
function fine in virtual 
environments because 
that’s what the business 
needs dictate.” 


—Brent Ozar, SQL Server Domain Expert at 
Quest Software 


SQL Server instance with the same care you do when 


installing to bare metal.” 


In his presentation “Virtualizing SQL Server using 
VMware Infrastructure” at VMworld 2008, Hemant 
Gaidhani, technical product marketing manager at 
VMware, provided some additional SQL Server vir- 
tualization tips. Gaidhani recommends placing the 
OS and application on separate spindles than the 
TempDB, presizing your data files, and managing 
your file growth manually—not by using SQL Server’s 
Autogrow feature. Gaidhani also encourages DBAs to 
move their TempDB files to a dedicated LUN, make 
sure that all TempDB files are roughly the same size, 
and preallocate TempDB with enough space to handle 


expected workloads. 


Virtualizing 
SQL Server 
Contrary to what some 
DBAs believe, virtual- 
ization and SQL Server 
can work together in 
harmony. Armed with 
a bit of research and 
some best practices, 
you can leverage vir- 
tualization to maxi- 
mize your investment 
in your SQL Server 
infrastructure. In these 
trying economic times, 
that might be the best 
practice to follow. Er 
InstantDoc ID 101516. 
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For more information about SQL Server virtualization, 
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“Demystifying SQL Server Virtualization,” 
InstantDoc ID 99841 

“Running SQL Server in a Virtual Server 
Environment,” InstantDoc ID 95799 

“SQL Mag Presses the Virtualization Hot 
Button,” InstantDoc ID 97845 
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“Virtualization Shootout: Hyper-V vs. ESX 
Server 3.5,” InstantDoc ID 99218 

“VMs vs. Multiple SQL Server Instances,” 
InstantDoc ID 97439 

“VMs vs. Multiple SQL Server Instances, 
Round 2,” InstantDoc ID 95148 
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SOL Server Basics! 


Join SOL MVP Allan Hirt on April 30, 2009 for 

“= fundamental SQL Server lessons tailored to pros new 

SL to SQL, plus live Q&A—all on your own computer! 

Learn how to plan, deploy, and administer SOL Server 
(includes aspects of SOL Server 2000, SOL Server 2005, 
and SQL Server 2008). Whether you're new to database 
administration or expanding your knowledge from another 
RDBMS, this series is your SOL Server solution. 
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Allan Hirt has been consulting, training, 
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and authoring books, whitepapers, and 
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10 years. Allan is the author of the 
E upcoming book Pro SQL Server 2008 
Failover Clustering (Apress), due to be published in the 
spring of 2009. 


Learn more about the speaker, sessions, 
and how to reserve your seat at: 
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Bytes from the Blog 
www.sqlmag.com/go/industrybytes 


NEC Enters the North 
American Server Market 

hile their timing may not seem the best consid- 

ering the current state of the economy, Japan- 
based technology manufacturer NEC has entered the 
US server market with a line of impressive new servers, 
including the record-setting NEC 5800/A1160. NEC 
has offered systems in Japan for many years, and has 
also acted as OEM for several other well known server 
brands in the United States. 

At a past PASS, NEC announced that their NEC 
5800/A1160, aka Monster Xeon Server, was the first 
server to achieve a mark of 1,400 tpsE on the TPC-E 
benchmark. This benchmark was achieved using a 12 
CPU (64-cores) NEC 5008/A1160 with 384 GB of 
RAM. The NEC system was running Windows Server 
2008 Datacenter x64 and SQL Server 2008 Enterprise 
x64. The 5800/A1160 is one of the first enterprise 
systems to make use of Intel’s new 6 core 7400 series 
of CPU’s (code-named Dunnington). The NEC 5800/ 
A1160 system can scale from 1 node, 4 sockets, 16 
cores, and 256 GB memory to 4 nodes, 16 sockets, 96 
cores, and 1TB memory. 

The NEC server line uses only Intel CPUs. Notably, 
NEC also offers an Itanium-based 5800/1000 server, 
which is something of a rarity today. The 5800/1000 
supports dynamic hardware partitioning, allowing hot 
add of processors and memory based on thresholds 
and policies of the running OS partition. In addi- 
tion, NEC offers Fault Tolerant Servers like the NEC 
Express 5800/300, which offers full internal hardware 
redundancy and provides 99.999 percent system 
uptime. The company also offers its 5800/100 series line 
of blade servers and the SMB-oriented 3800/100 line 
of rack and tower servers. 

While not well known in the United States, 
NEC offers an enterprise-ready server line that’s 
certainly comparable to IBM, HP, and Dell. You 
can peruse the models in NECs new server line at 
www.necam.com/servers. SQL 

—Michael Otey 
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Microsoft SQL Server 2008 

Places in the Gartner 

Leaders Quadrant 

M= announced that Gartner Research 
placed SQL Server 2008 in the Leaders 

Quadrant in their Magic Quadrant for Data Ware- 

house Database Management Systems, 2008. Other 
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enterprise-class data warehousing solution platforms 
included in the Quadrant are Teradata, Oracle, IBM, 
and Netezza. 

So what’s a magic quadrant? According to Gartner, 
“Quadrants . . . offer visual snapshots of a market’s 
direction, maturity and participants. Understanding 
our research methodology will help you use these 
models effectively when choosing a product or service, 
or managing a vendor relationship.” According to a 
Microsoft press release, the Magic Quadrant for Data 
Warehouse Database Management Systems (DBMS) 
depicts Gartner’s analysis of how data warehouse 
DBMS software-only vendors measure against specific 
criteria within the marketplace. 

The Gartner report says that “The data warehouse 
DBMS market has evolved from a traditional informa- 
tion store supporting business intelligence (BI) users 
and tools into an analytics infrastructure repository 
for the enterprise. And organizations are adding addi- 
tional workload from online transaction processing 
(OLTP) applications and increasing the frequency 
of data loading to intraday, approaching continuous 
loading.” 


“The data warehouse DBMS 
market has evolved from a 
traditional information store 
supporting business 
intelligence (Bl) users and 
tools into an analytics 
infrastructure repository for 


the enterprise.’ 
—Gartner Research 


The Microsoft press release explains that the post- 
RTM SQL Server 2008 project called “Madison,” 
which integrates DATAllegro’s massively parallel 
processing technologies, extends SQL Server’s data 
platform to support the largest BI and data ware- 
housing deployments. For more commentary from 
Microsoft, see the MSDN blog “Architects Rule!” at 
blogs.msdn.com/architectsrule/default.aspx. For more 


on Madison, see “Microsoft Unveils New Technologies 


at BI Conference,” InstantDoc ID 100485. SQL} 
—Sheila Molnar 
InstantDoc ID 101191 
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“What | had been fighting with for three or four days, you 
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DATABASE MANAGEMENT Q Editor’ Tip 
CA ERwin Data Modeling Suite Updates 


To celebrate its 20-year anniversary, CA is releasing a new version of its CA ERwin Data Modeling Suite, which Got a great 
includes new tools and updates. The first is CA ERwin Data Profiler 3.2, which provides data profiling to ensure new product? 
your data is consistent—that there is only one version of truth in your databases. Second, the company is releasing Send announce- 
the latest version of CA ERwin Data Modeler 7.3, which lets you share important information with various data ments to products @ 
management tools such as Oracle BI, Cognos, and SAP/MDM. CA ERwin Data Profiler and CA ERwin Data 
Modeler are available for $4,000 each. To learn more about the suite, visit www.ca.com/modeling. 


sqlmag.com 
—Jeff James, 


BACKUP AND RECOVERY Editor-in-Chief 
ca meta cee NL E Space and Time Savings with LiteSpeed” Engine for Oracle 


Quest Software announced the launch of the LiteSpeed Engine 
— |, e 


for Oracle. LiteSpeed for Oracle will give Oracle database 
administrators (DBAs) the same high-performance backup 
compression and encryption technology that it has offered in 
the SQL Server market for years via LiteSpeed for SQL Server. == 19,13 

‘ era 9.77 
According to a Quest Software press release, Oracle DBAs 
charged with handling secure backup and recovery strategies E ee 
on a lean budget can realize 70 to 90 percent compression with 
LiteSpeed, while maintaining complete control over the backup 
and recovery process. To learn more about the product, call 949-754-8000 or visit www.quest.com. 


DISASTER RECOVERY 

Minimize SQL Server System Failures 

Data-Rite Systems Group has released DR-Synch 3.0, a disaster recovery solution that monitors your servers 
and notifies you of problems. DR-Synch works by backing up your transaction logs regularly and automatically 
restoring them on a standby server. In the event of a server failure, you can switch your users over to the standby 
server. Once the primary server is back up and running, DR-Synch automatically syncs the logs no 


matter how long the server has been down. This version replaces WinZip with a new compression $ 
algorithm to compress data faster, and includes historical tracing and improved detection of server ™ 
failures. DR-Synch costs $8,000 for unlimited use of servers and databases on the same network. To 

learn more, call 212-697-0207 or visit www.dataritesys.com. z 
BUSINESS INTELLIGENCE 1 
Enhanced Reporting to Access and Display Your Data H 
FileMaker has announced the release of FileMaker Pro 10, the new version of its database soft- > 
ware. FileMaker lets you create reports and change the underlying data from within reports. The  { 
new version offers Script Triggers, which let you automatically launch scripts based on user actions. à 


FileMaker Pro 10 lets you create databases from existing CSV, Tab, Excel 2007, or Bento 2 files. With 
version 10, you can access and use data from SQL tables on various database platforms, including Fi le M a ker 
SQL Server 2008, Oracle 11g, and MySQL 5.1 Community Edition. Pricing for FileMaker Pro 10 E upmana Drg 10 
starts at $299. For more information, go to www.filemaker.com. 


AUTOMATION 

Automate Batch Job and Task Scheduling 

MVP Systems has released JAMS 4.5, which helps automate complex business processes, integrate “at Tro) j g K SL 
and get rid of unnecessary processes. According to MVP Systems, JAMS is the only batch job scheduling tool” - | E A 

built on the Microsoft .NET Framework. It works by automating mission-critical business processes across mul- 

tiple applications and platforms. Enhancements found in JAMS 4.5 include new reporting features, SharePoint 

support, an improved UI that now provides a “Ribbon” toolbar, AJAX-based web controls, and more. You can 

download JAMS 4.5 from www.mvpsi.com. SQL 
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“SQL Server Storage Options” (sqlmag.com/go/ 


' SQLStorageOptions): Manage the ever-increasing 


fame Data Access Options in VS 2008 


and the .NET Framework 3.5 


eeping up with data-access enhancements in 

Visual Studio (VS) is a full-time job. Here’s a 
quick look at six new data-access options in VS 2008 
and the .NET Framework 3.5 SP1: 


ADO.NET 3.5 

ADO.NET 3.5 supports SQL Server 2008 data types 
including date, time, hierarchical, geospatial, and 
FILESTREAM. It supports database connectivity 
using the native .NET Framework Data Provider for 
SQL Server but can also connect to SQL Server and 
other databases using the OLE DB Provider and the 
ODBC Driver. ADO.NET remains a core technology 
that Microsoft will continue to support. 


LINQ to DataSet 

LINQ to DataSet combines the new LINQ syntax 
with the ADO.NET DataSet object. LINQ is a data- 
access technology that lets you construct queries 
directly in Visual Basic (VB) or C#, eliminating the 
dual development methodology that ADO.NET uses. 
LINQ to DataSet uses the ADO.NET DataSet’s cache 
for schema and storage and is useful for building 
n-tiered web applications. 


LINQ to SQL 

LINQ to SQL lets developers directly access SQL 
Server 2008 and SQL Server 2005 databases. You 
create an object-level representation of the relational 
database, either manually or by using the SQLMetal 
tool (msdn.microsoft.com/library/bb386987.aspx) or 


storage requirement 
with tips from th 
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ical Advisor eBook, you'll learn about 
tages and disadvantages of DAS and SAN, 
tate disk (SSD) as a storage solution, which 
ge solution is best for different situations, 
‘ocess to automatically collect space- 
usage statistics for each database. 


If you’re interested in more resources, contact me 


for free at christan.humphries@penton.com. 


the Object Relational Designer visual design surface 
(msdn. microsoft.com/library/bb384429.aspx).Using 
the object-relational model you created and LINQ, you 
can then query and update SQL Server databases. 


ADO.NET Entity Framework 
Going beyond the capabilities offered by LINQ to 
SQL, the ADO.NET Entity Framework lets you create 
a full conceptual model across the enterprise. While 
LINQ to SQL is limited to SQL Server databases and 
provides a one-to-one mapping between relational 
database objects and program objects, ADO.NET 
Entity Framework’s allows data to be mapped to a 
more business-related object-oriented model. 


ADO.NET Data Services 

ADO.NET Data Services, previously code-named 
Astoria, addresses the problem of accessing your 
data online. It exposes an object model by using the 
REST protocol. Data can be consumed by a .NET 
Framework 3.5 client, by using Silverlight 2, and by 
ASP.NET AJAX clients. 


SQL Server Data Services 
VS 2008 developers can also use SQL Server Data 
Services (SSDS), essentially a data store in the cloud. 
Data is exposed using the REST or SOAP protocols, 
enabling SSDS to be accessed by .NET and other 
technologies such as Java that support these protocols. 
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SQLSENTRY } 
PerformanceAdvisor ° 


The New Standard in Monitoring and Performance 


Hierarchical display provides the most complete picture of blocks available, and block 
chain version-tracking lets you reconstruct the block for any point in time. 


Wait Resources are automatically resolved for you, so for the first time you can instantly 
see exactly which tables or indexes are being contended for without any manual effort. 


Generate alerts with full block chain details whenever a block occurs, when a block 
exceeds a specific duration, or when a blocking query contain specific text. 


Graphical display of all deadlock types, synchronized with an innovative grid-based 
display to provide the most complete picture of deadlocks available. 


Deadlock nodes contain the concise information you need to quickly visualize the 
deadlock and determine the root cause. 


Generate alerts including the deadlock victim SQL whenever a deadlock occurs, or 
when the victim query contains specific text. SQL Server Deadlocks 


Performance Dashboard with relevant SQLServer + Top SQL analysis highlights heaviest queries 


SNE bullets rnai Te e Graphical blocking and deadlock analysis 


Real-time and historical performance analysis 3 


Calendar views of Top SQL, blocks and deadlocks 


Disk activity, latency, and capacity monitoring e One-click and automated tracing with Quick Trace™ 


Free Trial Download: sqlsentry.net/sql-performance SENTRY 
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Let SOL diagnostic manager track down your SOL Server performance offenders! 


With SOL diagnostic manager, DBAs can quickly sniff out performance problems 
BEFORE they become a crime! 


With over 100,000 SQL Servers monitored worldwide, Idera leads the market in SQL Server performance and diagnostics solutions. 


Our products are easy to use and low-impact. What’s more, you'll be up and running in minutes! Www.idera.com 


